Personality Profile Assessment

[dlorde]

Apologies in advance for a lengthy post...

I've recently enountered a commercial version of Personality Profile Assessment (the Thomas International PPA System), and being sceptical of the underlying science and methodology, and unable to find much solid information on it online, I'm curious to know the opinion here.

For those who aren't familiar, it purports to provide some insights into an individual's personality characteristics in terms of DISC (Dominance, Influence, Steadiness, & Compliance). These measures were developed in the 1920s as a way to model behavioural variations between individuals in their response to their environment. Neglected for many years, it was revived in the 1980s and a number of companies now seem to be offering personality assessments, for corporate use, based on variations on this theme.

The system I encountered proceeds by giving 20-odd questions, all of the same form: the subject must select from four orthogonal personality characteristics the one most like and the one least like their behaviour in the work environment. For example, the first question provided these options:

Gentle, Persuasive, Humble, Original

- I find these questions nigh-on impossible to answer - am I more gentle than original, or persuasive, etc., and am I less humble than gentle..??

Once completed, the answers are automatically processed to produce a profile graphing the relative DISC measures for 'Work Mask' (the actual behaviour at work), 'Behaviour under Pressure', and 'Self Image'. As I understand it, the 'Work Mask' is derived from the 'most like me at work' selections, and the 'Self Image' from the 'least like me at work' selections. On what basis, I have no idea.

In addition, an extremely detailed 'written' profile (3 pages of A4) is also generated. For example:

"Behaviour Under Pressure:
Whilst the self-image suggests that X is steady, thorough, and dependable, all the indicators suggest that he may have a mercurial change in behavioural style. Initially when pressure is placed upon him, he increases his natural characteristics, but if continued extreme pressure is exerted, he is likely to reverse his behaviour. As a result he may be seen as restless, even anxious. He may also be seen as somewhat volatile. Although this would be only a temporary situation the change of style should be assessed."

- I note that the comments in this section in particular will apply to almost anyone put under extreme pressure...

These profiles are being used by large corporations as an adjunct to interviews, and as a way of helping assess staff suitability for particular roles.

Now I am prepared to accept that by correlating a large sample of such questionnaires with detailed traditional personality assessments of the same subjects by trained interviewers, it may be possible to assert that an individual PPA can indicate DISC characteristics with a certain degree of confidence, but I can't find any evidence of such research - the tests seem to have been thoroughly analysed for self-consistency, and consistency with other types of personality assessment, but the only face validation I could find was a sample of 1500(?) students who were asked to rate their own profiles - and rated them over 80% accurate (although I guess palmistry or tarot readings could match that).

What I find particularly difficult to accept is the more detailed analysis and how it could be generated from the answers.

The disclaimer is that the results are only a 'guide', and 'should never be used in isolation', but how can such a specific and detailed comment as "Ideally, regular or unplanned travel should not be a part of the job." be generated from the 20-odd sets of answers? what algorithms can be involved?

My gut reaction is GIGO - garbage in, garbage out.

Apparently user satisfaction is very high, and these profiling companies are very successful, but it seems to me that they're onto a nice little earner.

When I mentioned my concerns about the lack of scientific or statistical basis for these assessments to a software manager who'd been on a PPA training course, I got all the stock marketing responses, but discovered that the course hadn't covered the origins of or statistical/scientific basis for any of it. Indeed, my querying of the science and rationale was exactly what my profile would predict! His unquestioning faith was remarkable.

It disturbs me that these assessments are being used in the workplace. Despite the disclaimers, the 'written' profiles are far from vague and general, they are quite detailed and specific - how long before they are seen as a convenient way to avoid interviews altogether, or move staff to different roles without the effort of consultation?

Anyone have comments or experience of these things? are my concerns well-founded? Is this just another scam, or is there some scientific basis after all?

[Cuddles]

Originally Posted by dlorde

Anyone have comments or experience of these things? are my concerns well-founded? Is this just another scam, or is there some scientific basis after all?

Scam. There are millions of this sort of personality test around on the interweb, mostly giving a free sample and asking you to pay for your full 10 page assessment by an "expert". Like all kinds of cold reading and horoscopes, the user satisfaction is high beause they say nice things about people. Obviously a company that said people were complete ********* with no chance of being competent in their job wouldn't get many customers.

Quote:

The system I encountered proceeds by giving 20-odd questions, all of the same form: the subject must select from four orthogonal personality characteristics the one most like and the one least like their behaviour in the work environment. For example, the first question provided these options:

Gentle, Persuasive, Humble, Original

- I find these questions nigh-on impossible to answer - am I more gentle than original, or persuasive, etc., and am I less humble than gentle..??

Like you say, the questions are meaningless and impossible to give a sensible answer to. In this example the four are certainly not orthogonal, it is perfectly possible to be all four at the same time. Again as you say, garbage in, garbage out. The questions are meaningless, the answers are meaningless and therefore any "analysis" of the answers is meaningless.

[The Don]

I've taken a number of these kinds of tests over the years and they've produced consistent answers. They've all described the way I'd like to be and behave rathar than the way I am

[benji]

Originally Posted by The Don

They've all described the way I'd like to be and behave rathar than the way I am

Yes, but don't you think your answers concerning they way you'd like to be and behave tells something about you too? Only a novice Psychologist (or scam artists like the ones mentiones above) would interpret the result of a (scientific) personality questionnaire as "how you are". At best, the results say something about how you see yourself. But together with other data (supportive or contradictory), that information may be very valuable...

[dlorde]

Quote:

At best, the results say something about how you see yourself. But together with other data (supportive or contradictory), that information may be very valuable...

Possibly, but these assessments claim to describe the way you are and the way you would behave in various situations. If there is no basis for this claim, it seems to me more dangerous (in corporate hands) than valuable.

[Brian Jackson]

Do the Myers-Briggs test. It's more accurate and widely used... and free.
http://www.humanmetrics.com/cgi-win/JTypes1.htm

(I'm an INTJ)

[benji]

Originally Posted by dlorde

Possibly, but these assessments claim to describe the way you are and the way you would behave in various situations.

No they don't. At least scientific ones don't (read any handbook of a scientific personality inventory). That's exactly why these instruments should only be used by certified personnel.

Originally Posted by dlorde

If there is no basis for this claim, it seems to me more dangerous (in corporate hands) than valuable.

True. They may become dangerous in the hands of unqualified people - unfortunately, most of the people who use these instruments nowadays are unqualified (and most of the instruments available online are just garbage).

[Skibum]

Originally Posted by Brian Jackson

Do the Myers-Briggs test. It's more accurate and widely used... and free.
http://www.humanmetrics.com/cgi-win/JTypes1.htm

(I'm an INTJ)

I've taken that test several times, I come up as either ISTJ or INTJ every time.

[Fnord]

Originally Posted by Brian Jackson

Do the Myers-Briggs test. It's more accurate and widely used... and free.
http://www.humanmetrics.com/cgi-win/JTypes1.htm

(I'm an INTJ)

I'm an ISTJ this time. It's also called an "Inspector Guardian" and such people supposedly "look carefully and thoroughly at the people and institutions around them. Making up perhaps as much as ten percent of the general population, Inspectors are characterized by decisiveness in practical affairs, are the guardians of institutions, and if only one adjective could be selected, "superdependable" would best describe them. Whether at home or at work, Inspectors are nothing if not dependable, particularly when it comes to examining the people and products they are responsible for-quietly seeing to it that uniform quality is maintained, and that those around them uphold certain standards of attitude and conduct."

(Yeah ... right ... and I'm doing this while I should be working instead.)

My top four most suitable occupations are:

- Technical
- Computer Programming
- Technology Education
- Engineering

Okay, at least that much is valid. Those four occupations describe most of what I do (when I'm not posting in a JREF forum).

Great fun. Now, back to work!

[ChristineR]

I usually come out as INFJ, and the descriptions of the INFJ do not describe me at all.

[JJM]

The Myers-Briggs test is as reliable as astrology, phrenology, palm reading, etc.:
http://www.skepdic.com/myersb.html

When I was a kid they gave us the "Kuder Preference Test" which had multiple choice questions like- Would you rather
A) Supervise sugarbeet workers
B) Weave a rug
C) Drive race cars

My results came back that I should be a catholic priest. I thought that sounded good, you work one morning a week in return for three hots and a cot. Then they told me what "celibate" means ...

[strathmeyer]

You could always sit in on a few personality classes at a local university for free. If you're tricky enough you can even take the tests...

[Almo]

Originally Posted by Brian Jackson

Do the Myers-Briggs test. It's more accurate and widely used... and free.
http://www.humanmetrics.com/cgi-win/JTypes1.htm

(I'm an INTJ)

I like this one too. INFJ. Knowing other people's type from this test have actually helped me to interact with them in a more meaningful way.

[JJM]

Originally Posted by ChristineR

I usually come out as INFJ, and the descriptions of the INFJ do not describe me at all.

You're a tough customer- 80% of people are happy with any determination http://www.skepdic.com/forer.html. Didn't they tell you that you can request another designation?

[JJM]

Originally Posted by Almo

I like this one too. INFJ. Knowing other people's type from this test have actually helped me to interact with them in a more meaningful way.

Well, I didn't know Myers-B had anecdote supporting it ...

[ChristineR]

Some of the INFJ description is in fact accurate, and I'm surprised that only 80% find their descriptions accurate. So much of that test is just feeding your own answers back to you.

[JJM]

Originally Posted by ChristineR

Some of the INFJ description is in fact accurate, and I'm surprised that only 80% find their descriptions accurate. So much of that test is just feeding your own answers back to you.

That was 80% find any M-B description accurate. You are right, many more find their "personal" attributions correct.

[Almo]

Sorry man. I've really gotten good results. The base types: NF (McCoy), NT(Spock), SP(Kirk), SJ(Scotty) are kind of obvious, and different people really do seem to fall into these categories. My wife is an NF, her mom is an NF, but her dad is an NT. It shows very clearly in their behavior.

When I read astrology stuff, I see myself in all of them. But with the MB, I don't see myself in most of the types, using "Please Understand Me" as a reference. ENTP? No way man. ISTJ? No way either.

As just a simple example, the E/I section of the test focuses on your interactions with others; whether you are invogorated by interacting with loads of people at once, or just a few. The questions ask if you are first to answer the phone, if you leave parties with more or less energy, etc. This attempts to figure out if you're an introvert or extravert.

Yup, it's all anecdotal. Doesn't prove a thing. Next subject.

[JJM]

Originally Posted by Almo

Sorry man. I've really gotten good results.

[Yada Yada Yada.]

Yup, it's all anecdotal. Doesn't prove a thing. Next subject.

Huh?

[dlorde]

Well I took the Myers-Briggs, and came out INTJ - and I thought: tell me something I don't know...

So opinion seems to be that Myers-Briggs has something, but the others are crap?

[Warge]

How about asking the question: For what purpose do we do personality tests? And can they exactly tell how we are? In my work as a career counselor, I have come accross several types of personality tests (mostly Holland's however) in order to see what an applicant would be suitable to work with. I never use these tests however since I'm of the opinion that tests are only there to push people into little boxes.

We are about 6 000 000 000 people on the earth today; in my opinion that means there are 6 000 000 000 personalities. However, grouping people together according to personality type, race, sex, nationality, home town etc is natural. We must do that in order to be able to grasp the fact there are so many people.

However, basing any solutions, suggestions or say career choice upon what a few questions and answers have said is horrendous.

A better answer can be found at socionics dot com slash articles slash fpt dot htm

Funny thing this btw...

[bpesta22]

Ah, Warge, I believe testing is the only real contribution psychology's made to humanity. No where is this more evident than in tests used for employment decisions and/or career development.

A 5 minute measure of conscientiousness has better validity than an unstructured interview conducted by a seasoned human.

I'm of the opinion the MBTI sucks too, for the reasons listed in the wiki link above.

the NEO's the way to go

All my opinion, but I can provide cites if anyone's interested.

Warge-- just curious your philosophy re career development, if you don't mind sharing.

Thanks!

[tkingdoll]

Ask yourself this: could you have skipped all the questions and just been given a list of personality types from which to pick the one that best describes you?

[JJM]

Originally Posted by dlorde

... So opinion seems to be that Myers-Briggs has something, but the others are crap?

No. Read the links I posted- Myers-Briggs is crap, too. If people here have been fooled by it, it doesn't matter.

[Gwyn ap Nudd]

Once, when I had nothing better to do, I worked out the corresponding MB profiles to all the astrological signs based on the assumptions that earth=ST, air=NT, water=NF, fire=SF and cardinal=EJ, fixed=IJ, mutable=IP

So, Aries (cardinal fire) becomes ESFJ; Taurus (fixed earth) becomes ISTJ, etc. The MB profiles and the astrological profiles were very close.

[Gwyn ap Nudd]

Originally Posted by JJM

That was 80% find any M-B description accurate. You are right, many more find their "personal" attributions correct.

Only 80%? The Forer effect (the equivalent of the placebo effect when testing personality scales) is 84%. That means that MB does not even predict as well as a random assessment!

[benji]

Originally Posted by tkingdoll

Ask yourself this: could you have skipped all the questions and just been given a list of personality types from which to pick the one that best describes you?

It's not THAT easy.

Say you describe yourself as "very passionate". Where does this image of yourself come from?

It implies that you have an image of the rest of mankind, and that image is one of a "on-average-less-passionate-than-you" humanity.

It has been shown that 80% of car drivers describe themselves as above average" drivers. Wrong self-image, or wrong "rest of drivers"-image?

The idea of a test is to present standardized situations and ask for your attitudes towards them. The simpler these questions are (e.g. "do you like to go out?"), the easier they are to answer. The more "abstract" they are(e.g. "do you think of yourself as an honest person?") the harder they are to answer reliably, since room for subjective interpretation increases.

The real value of a test (or rater "inventory", since you can't "pass" a personality instrument) is the oportunity to compare your attitudes against the answers of a sample population.
That's why a standardized test (if well done) will always get better results than an interview or just an "impression" where you have to rely on your subjective "feeling". And of course, a test should only be ONE element in an assessment - not THE only one. It's a source of information that is not easily accessible otherwise.

[dlorde]

Originally Posted by tkingdoll

Ask yourself this: could you have skipped all the questions and just been given a list of personality types from which to pick the one that best describes you?

Yes indeed. I'm sure a little introspection and some honesty could provide a far more accurate profile. As I said to the PPA enthusiast, if I were to answer the same questions again, I'd probably answer 2/3rds of them differently - not because my opinions had changed, but because the questions were so difficult to answer. I would have difficulty rating myself for each characteristic on an arbitrary scale (humbleness 6/10?, gentleness 7/10?, etc.) let alone comparing these characteristics against each other without any scale - are they linear? does consistent slight persuasive ability outweigh rare flashes of stunning originality?

If you want to know how I behave at work and how I'd like to behave at work, ask me. I agree that it would be prone to personal biases, and not standardised, but I'm contrasting it with a test where the input is (appears) to be largely arbitrary, if not random. I'd also agree that a standardised test with simple questions would probably be more useful and reliable than either.

[benji]

Originally Posted by dlorde

Yes indeed. I'm sure a little introspection and some honesty could provide a far more accurate profile.

maybe, maybe not. See "drivers" above. What is missing is an objective point of reference.

Originally Posted by dlorde

As I said to the PPA enthusiast, if I were to answer the same questions again, I'd probably answer 2/3rds of them differently - not because my opinions had changed, but because the questions were so difficult to answer. ...

Exactly. That test works on a very "abstract" level and as such ist almost impossible to answer reliably.

Originally Posted by dlorde

If you want to know how I behave at work and how I'd like to behave at work, ask me.

Actually, the best way would be to observe you (unknown to you). But since that 's impossible to do on a large scale (not speaking about other problems this aopproach would have), the next best thing really is to ask you. Which is exactly what (good) personality inventories do. Plus, most important, they compare your answers to those of others.

Originally Posted by dlorde

I agree that it would be prone to personal biases, and not standardised,

Your "personal" biases COULD mean that your answers are completely unrealistic. You could have the impression of being a very hard-working person compared to your co-workers (after all your self-image always stems from a comparison to something or someone), while in reality you are the laziest guy in the office. Humans are world champions in self-deception. Asking standardized questions on a more basic level may eliminate part of that error. It's still a self image that is assessed, but at least it is compared objectively to other self-images.

Originally Posted by dlorde

but I'm contrasting it with a test where the input is (appears) to be largely arbitrary, if not random. I'd also agree that a standardised test with simple questions would probably be more useful and reliable than either.

I see we agree

[tkingdoll]

benji, how do you account for people giving the answers they think the employer wants to hear? I sincerely doubt there is much honesty in these assesments as a recruitment tool.

The other issue with them is that there is no evidence supporting the company's eventual choice of candidate being the right one as a result of the profile.

[Rinky]

Can't really say if my M-B analysis was 'accurate' or not:

"Sorry, there is not enough data for your type analysis"

Should I be worried?

[benji]

Originally Posted by tkingdoll

benji, how do you account for people giving the answers they think the employer wants to hear? I sincerely doubt there is much honesty in these assesments as a recruitment tool.

An important problem that causes ongoing discussions. In the early days of personality assessment, faking your answers meant that the results of a test (let's just stay with this expression even if its wrong. It's just shorter than writing "inventory" every time) should not be interpreted.

Some newer approaches regard the ability to fake in a way that is actually desired (to fake successfully, you have to know what people expect of you) as a personality trait by itself.

However, detecting faked (socialy desirable) answers is actually not that hard. It's easy to fake your answers in a certain direction, but it is a lot harder to control the extent of your faking. Getting desirable but still believable resultes by faking is actually pretty hard to do. And then - as I said - a personality test should never be used as the only mean for a selection. Getting an extremely "positive" profile that contradicts other data doesn't necessarily mean that someone actively tried to fake, but it could be interpreted as having an inadequate self-image. Which isn't really a huge bonus if you want to get a certain job.

If you make these things clear befor administering a tzest (in nice words of course - talking about the best job-person-match and things like that) , people are a lot less motivated to fake in the first place.

Originally Posted by tkingdoll

The other issue with them is that there is no evidence supporting the company's eventual choice of candidate being the right one as a result of the profile.

Again - the choice of a candidate shouldn't depend just on one profile. It's one element in the process of reaching a decision. But you are right that companies can't really evaluate their decisions - which is true for any method that leads to a decision. This makes it even more important to rely only on scientific instruments that have been proved to be of value for the task of answering certain questions in studies before (reliability, validity).

It's clear that with all the subjective influences, a personality test can never be as reliable as an aptitude test, but sometimes there is no other (better) way to get the information you need to answer a certain question.

(Ooops - somehow these last sentences sound bad, but I hope you understand what I mean. English isn't my first language)

[roger]

Why MTBI is crap, by roger XXXXX
Proof through example

There are four elements in the world: earth, wind, water, fire. These elements pervade everything, including people. Everything and everyone is made up of either one or more of these 4 elements, and the mix of the properties determines how you or it reacts. For example, a cactus is made of earth, with a bit of fire. It sits in the sun all day, making it hot, and if you brush against it you will surely feel the effect of the fire element!

It's the same with people. Take someone made of wind. You know these people - they blow every which way, they cannot make up their mind, they also go with the prevailing opinion. Fire - that's easy. A fire-y person is loud, vivacious, quick to anger and laugh.

You probably know people with a mixture of the 4 elements. Perhaps you know somebody who is quick to laugh, is very vivacious but is able to see other's points of view, and thus doesn't fly off the handle. This person is mainly fire, but tempered with earth, which grounds them.

There is so much more to learn. For the next hour, you can buy my book for low low payments of $5.95 each (for 24 months). Order now!!!!


---------------------
Okay, the above is completely sound in one sense, but completely bogus in another. I'm sure I got the details wrong, but it's the basic chinese 4 elements. They are used successfully to describe personalities. We could easily devise an instrument that measures what 4 elements you are made of. Anyone who knew you could read the results and largely agree with the descriptions. That's the sound part - we can make an instrument that measures us against 4 axis and describe the person in a way that they would recognize and agree with.

But, of course, it's bogus. We aren't made of 4 elements, our personalities are not split in 1 of 4 'types', etc. It's complete, and utter fiction. It has no real predictive capability (beyond "a fire person will get mad a lot"), it relates to no biological system, it's utter crap.

Likewise, there is no evidence for the MTBI 4 axis. It's bad science. As another person pointed out, you could easily pick out your "personality type" just by reading the descriptions.

And that's why we say MBTI is crap. Make a bunch of differing personality descriptions, make up some phony 'reasons', whether it be 4 elements, Jung's 4 personality archtypes, the 8 dwarves, whatever, and then make an instrument to "measure" them. I guarantee people will agree with the descriptions.

As for me? Well, I'm a SGD - Sleepy Grumpy Dopey. "OMG" my girlfriend exclaims, "that's just scary it's so accurate!!!"

[dlorde]

Originally Posted by benji

I see we agree

Yes, pretty much.

I suppose I was after a bit of validation for the strong opinions I expressed about that particular PPA methodology in the face of what came across as a cult-like conviction in the validity and usefulness of the results. It occurs to me that having invested the time, money, and effort in attending a course on the PPA with it's selective and dubious 'facts', and easy shortcuts to the 'real you', the PPA enthusiast was unlikely to be swayed by me beating him about the head with the rolled-up newspapers of scientific evidence, statistics and logic

Ho hum.

[tkingdoll]

benji, may I ask your opinion of graphology as a recruitment assessment method?

[benji]

Originally Posted by dlorde

Yes, pretty much.

It occurs to me that having invested the time, money, and effort in attending a course on the PPA with it's selective and dubious 'facts', and easy shortcuts to the 'real you', the PPA enthusiast was unlikely to be swayed by me beating him about the head with the rolled-up newspapers of scientific evidence, statistics and logic

Ho hum.

Anyone proposing "easy shortcuts" in "science" should be "beat about the head with the rolled-up newspapers of scientific evidence, statistics and logic"

Go get them

[Cuddles]

Originally Posted by benji

It's clear that with all the subjective influences, a personality test can never be as reliable as an aptitude test, but sometimes there is no other (better) way to get the information you need to answer a certain question.

To answer what question though? Do you have any evidence to show that personality tests do actually give the best answer to any questions?

[benji]

Originally Posted by tkingdoll

benji, may I ask your opinion of graphology as a recruitment assessment method?

Sure you may. It's extremely low. Let's say... zero.

There are experienced graphologers that can find out a lot from the material they are presented with - as long as the person to be assessed is present too. They have gathered a lot of experience assessing people (they built their own internal, representative database, so to speak), but that has nothing to do with graphology as a method (even though they may not even know this themselves).

A really good diagnostician may get results with any tool. But even (or especially!) good diagnosticians should be required to use good, professional tools, because even their results will improve.

Before you ask - I put graphology in the same basket as astrology, homeopathy,... (opinion-rating-wise)

[tkingdoll]

Originally Posted by benji

Sure you may. It's extremely low. Let's say... zero.

There are experienced graphologers that can find out a lot from the material they are presented with - as long as the person to be assessed is present too. They have gathered a lot of experience assessing people (they built their own internal, representative database, so to speak), but that has nothing to do with graphology as a method (even though they may not even know this themselves).

A really good diagnostician may get results with any tool. But even (or especially!) good diagnosticians should be required to use good, professional tools, because even their results will improve.

Before you ask - I put graphology in the same basket as astrology, homeopathy,... (opinion-rating-wise)

Just checking

[catbasket]

About six years ago we all at work (IT department of a multi-national insurance company) had to take the Myers-Briggs test. The research I did at the time convinced me the M-B was woo dressed as science.

(No links, no cites, take with massive pinch of salt)

I still believe the M-B test are woo, but as ever prepared to be shown to be wrong.