Moore's Law still holding on

[This Guy]

OK, so it was more of a hypotheses, I believe, than a law, or even a theory (Seems I read a quote from Moore that it was more a guess, but can't swear to that, and it was only expected to hold true for 10 years or so) -

The term Moore's Law was coined by Carver Mead around 1970.[4] Moore's original statement can be found in his publication "Cramming more components onto integrated circuits", Electronics Magazine 19 April 1965:

“ The complexity for minimum component costs has increased at a rate of roughly a factor of two per year ... Certainly over the short term this rate can be expected to continue, if not to increase. Over the longer term, the rate of increase is a bit more uncertain, although there is no reason to believe it will not remain nearly constant for at least 10 years. That means by 1975, the number of components per integrated circuit for minimum cost will be 65,000. I believe that such a large circuit can be built on a single wafer.[1]

http://en.wikipedia.org/wiki/Moore's_law



But it looks like it's going to be valid for a few more years anyway -

http://www.technewsworld.com/story/innv/55441.html

"Moore's Law, which postulates that the number of transistors on a chip will double every 18 to 24 months, recently faced a major roadblock: power leakage. However, by using so-called "high-k" materials, IBM and Intel both say they have remedied this efficiency problem, allowing the continued shrinking of computer chips."

I'm just waiting for my HAL on the wrist! ;)

[jimbob]

Yes, but with gate lengths of only several hundred Å you are getting towards the level where you have to stop treating the device as a "lump" of silicon with an average doping level, and start treating it as a collection of atoms...

It's all going quantum... (misquote of Terry Pratchett)

Power dissipation and power density is another issue: expect more asynchronous logic (no clock pulse to use up power)

3-D packaging could be interesting, (see problem about heat dissipation) as well as organic logic (larger area's but prionted where silicon wouldn't be...

I believe it is slowing down:

Also one of the main drivers for process research (the Crolles Alliance) is now falling away (with two of the three main players pulling out) (NXP, ex Philips, and Freescale, ex Motorolla).

That leaves TSMC and IBM probably.

You want how many billion for a new fab???

Jim

[andyandy]

what's the current number of transistors on a chip we can do?
How many angels are dancing on our pinheads? :)

[jimbob]

working on 45nm (450Å) gate lengths...

The high-k (dielectric) materials make efficient capacitors (good for dram) and can make higher-gain transistors. Short-channel effects are important (leakage, which is why the "FINFET" is getting to be important:

Jim

[This Guy]

working on 45nm (450Å) gate lengths...

The high-k (dielectric) materials make efficient capacitors (good for dram) and can make higher-gain transistors. Short-channel effects are important (leakage, which is why the "FINFET" is getting to be important:

Jim

Was this an answer to Andy's question about the number of transistors on a chip?

I'm so confused.

I actually tried to find something (anything) that actually stated the number of transistors per inch, or chip, or anything, and kept coming up with words like "Increased transistor density" or 40% increase, all nice information, but no actual transistor counts.

Anyone have a link or listing for the number transistors for say the PI, PII, PIII and P4 processors, for comparison?

I'd love to see something like that. But appears I'm too stupid to find it :crazy:

[jimbob]

Yes it was, one orang looks like another to me, sorry

Can't be bothered to work out the densities from the reciprocal length...

I'm a lazy git

Jim

[This Guy]

Yes it was, one orang looks like another to me, sorry

Can't be bothered to work out the densities from the reciprocal length...

I'm a lazy git

Jim

lol

Naa

I'm just not smart enough to figure it out. I was hoping you were talking about something else, cause I feel even more dumber now ;)

[andyandy]

here we go....intel has a "fun facts on 45-nm transistors" :)


Fun facts about 45-nm transistors
Hundreds could fit on the surface of a single red blood cell
2,000 fit across a human hair
30 million fit on the head of a pin
It can switch on and off approximately 300 billion times a second http://www.intel.com/technology/silicon/45nm_technology.htm


and here's there pdf fun facts sheet http://www.intel.com/pressroom/kits/45nm/Intel%2045nm%20Fun%20Facts_FINAL.pdf

wow.....30 million on a pin head......:jaw-dropp

[Ben Tilly]

"Moore's Law, which postulates that the number of transistors on a chip will double every 18 to 24 months, recently faced a major roadblock: power leakage. However, by using so-called "high-k" materials, IBM and Intel both say they have remedied this efficiency problem, allowing the continued shrinking of computer chips."

I'm just waiting for my HAL on the wrist! ;)

Moore's Law has several variations and their implications are more interesting than most people realize.

First of all outside of the IT industry nobody seems to have noticed that one of the key forms of Moore's Law has ended. It used to be that transistor counts and clock speeds both doubled quickly. But clock speeds maxed out a while ago and will not improve substantially in the forseeable future. Instead Intel and AMD are moving to putting more cores on one chip. In essence instead of getting faster CPUs, you'll be getting more CPUs instead.

There is much talk about our being forced to new programming paradigms to take advantage of this. Of course for half of what I do (web programming) there is no issue - there are always so many requests running concurrently that we can use any number of CPUs with naive parallelism. And for the other half (developing reports that run on a relational database) it is officially someone else's problem. (Namely Oracle's.) But there is no question that more programmers will need to worry about issues of concurrency and parallelism in the future. But I'm not (currently) one of them.

On another side note, it has been discovered that Moore's Law is not as exceptional as had been thought. It has been documented that in a wide variety of industries with agreed upon measures of success there is constant improvement. And normally the improvement is usually approximately exponential in time for an extended period of time. (Though admittedly it is slower than Moore's Law.) So even after Moore's Law has failed, there will still be a lot of variations of Moore's Law available.

In fact there are other examples of Moore's Law like phenomena in computers. All of the following show exponential improvement in time:

Number of transistors per chip
Clock rate (this one stopped improving recently)
Density of data in RAM
Density of data on hard drives (this one grows faster than other variations of Moore's Law)
Rate of data transfer to/from hard drive (this one is slow)
Latency of hard drive (ie time for a round trip) (this improves very slowly)These are all important in technology. But in many ways the fact of improvement is not as important as the relative rates of improvement. Because those relative rates determine where the bottlenecks are going to show up.

For example it used to be smart to have a certain amount of data in RAM, and then have a swap partition with more space for data. This made sense when RAM cost a lot. However if your machine actually has to go to swap, then it has to make a lot of unpredictable seeks to specific items of data. Given the latency of hard drives, this is relatively very slow, and getting slower all the time.

It still is standard to reserve space on a hard drive for swap. But knowledgable people when configuring a server tend not to do that. Instead they'll buy extra RAM and not have any disk reserved for swap. As the tradeoff gets more dramatically worse, that's going to be a more and more common choice going forward.

Cheers,
Ben

[Jimbo07]

As the tradeoff gets more dramatically worse, that's going to be a more and more common choice going forward.



On three machines in a row now, I have achieved a more profoundly noticeable performance boost by adding RAM rather than the attendant (or in some cases subsequent) processor upgrade.

[jimbob]

Was this an answer to Andy's question about the number of transistors on a chip?

I'm so confused.

I actually tried to find something (anything) that actually stated the number of transistors per inch, or chip, or anything, and kept coming up with words like "Increased transistor density" or 40% increase, all nice information, but no actual transistor counts.

Anyone have a link or listing for the number transistors for say the PI, PII, PIII and P4 processors, for comparison?

I'd love to see something like that. But appears I'm too stupid to find it :crazy:

OK, I've got the PC for a while...

assuming a 45nm gate transistor is, with its source and drain ~50nm, so (1/50e-7 per cm) or 500,000 per inch.

Of course you need to attach the "wires" and the rest of the circuitry and worh out how wide you want your transistors...

You also neeet to realise that you are making these on a 12" slice of silicon, and need to mace an economic number, so need to keep the number of defects down, and have enough of the transistors operating in the right part of the bellcurve for them to perform at speed. (The Intel P90 and P75 chips were apparently the same design, just the ones that were on the better part of the distribution (as tested) were turned into P90's and the slowere ones into P75's)

You can sometimes design in some fault tolerance, at the expense of silicon area, so

(It's complicated, but interesting, is the short answer)

BTW Do I have to wait for 50 posts before I can put up my avatar? I currently get an option to "not display an avatar" but none to display one...

Jim

[Ben Tilly]

On three machines in a row now, I have achieved a more profoundly noticeable performance boost by adding RAM rather than the attendant (or in some cases subsequent) processor upgrade.

Oh that has been standard advice for a long time now. When configuring a machine, add extra RAM and hard drive. The difference in performance between a top CPU and mediocre CPU is not very big. The difference in performance between a machine that has run out of memory and one that has not is infinite. And, as I pointed out, having to go to swap really hurts (and will only hurt more going forward). Therefore having extra RAM and hard drive will result in a machine that lasts better than one where you put that money into the CPU instead. (Unless, of course, the machine breaks.)

Cheers,
Ben

[jimbob]

The amusing thing is that IBM recently was getting out of hard drives, google the IBM centipede: it's back to basics for them, a punched card machine (on the nanoscale).

Well I find it amusing.

The cache-RAM-Swap optimisation is simply a means of getting the most frequently used data as close (in space and thus in delay time) to the processor as possible. cache could have ben SRAM, on the processor so no need to refresh it unlike DRAM...


Jim

[lenny]

Moore's Law has several variations and their implications are more interesting than most people realize.

have the "new" developments of this thread been included in forecasts of compute power over the next 30 years or so? i have an old DARPA slide (changes in cpu power from 1990 till 2030-ish) but need a new projection (DARPA, IBM, anyone reasonable will do!).

thx

[Schneibster]

It is worth noting for the non-cognoscenti that the speed of light is about a foot per nanosecond. This becomes significant when you're running three or four CPU cycles per nanosecond, as you are at 3 or 4 GHz.

[Zep]

On three machines in a row now, I have achieved a more profoundly noticeable performance boost by adding RAM rather than the attendant (or in some cases subsequent) processor upgrade.'Twas always the way. We knew this back in the 70's, when we were configuring *gasp!* minicomputers running swapping OS's (notably the first UNIX systems).

Of course, RAM was FRIGHTFULLY expensive then, so the much more cost-effective solution (yet to be discovered by Windows coders, incidentally) was to make your code much smaller and more efficient, so it didn't have to swap at all. It cost way less to pay a programmer (such as me!) for a month to write good small tight code that ran orders of magnitude faster than to buy a chunk of RAM that gave only a small percentage gain.

Funny - that still holds true today... :boxedin:

[Tony]

Lets cut the crap, when can I expect a 100 terabyte machine?

[Schneibster]

Lets cut the crap, when can I expect a 100 terabyte machine?You can have one right now. Cost a bit, though- a terabyte of disk is going for what, a few hundred simoleans? Call it five hundred. So that would be $50K. Plus the box to store stuff on it and pull it off.

[Schneibster]

'Twas always the way. We knew this back in the 70's, when we were configuring *gasp!* minicomputers running swapping OS's (notably the first UNIX systems). Indoody.

Of course, RAM was FRIGHTFULLY expensive then, so the much more cost-effective solution (yet to be discovered by Windows coders, incidentally) was to make your code much smaller and more efficient, so it didn't have to swap at all. It cost way less to pay a programmer (such as me!) for a month to write good small tight code that ran orders of magnitude faster than to buy a chunk of RAM that gave only a small percentage gain.The Russians could only get like 286s and stuff, because of the technology embargoes, while we were playing with pentiums, so they did stuff like write something in C, then compile it, then DISASSEMBLE it, optimize the assembly language, and reassemble. Tetris was written that way, rumor has it.

Funny - that still holds true today... :boxedin:Well, I suppose it does some places, for some tasks. Obviously not so much for the Microserfs.

The problem with such code bloat, of which they have been repeatedly accused, is that it means that there is a lot of code that has only been seen a few times. If you go over a piece of code enough to make it tight, you KNOW it. Nobody knows the code in Windoze like that, at least not most of it. Which is why they have so many damn bugs and security holes and crap.

[jimbob]

Lets cut the crap, when can I expect a 100 terabyte machine?


The engineer( http://www.e4engineering.com/Articles/296721/Billion+dollar+brains.htm) (another nice "comic" that I enjoy, although they are sometimes stonkingly poor on electroncs, confusing PCB substrates with silicon wafers in one memorable occasion) : was talking about petaflop computers... for supercomputers. In Japan. By 2012.

[Ben Tilly]

Of course, RAM was FRIGHTFULLY expensive then, so the much more cost-effective solution (yet to be discovered by Windows coders, incidentally) was to make your code much smaller and more efficient, so it didn't have to swap at all. It cost way less to pay a programmer (such as me!) for a month to write good small tight code that ran orders of magnitude faster than to buy a chunk of RAM that gave only a small percentage gain.

Funny - that still holds true today... :boxedin:

Who does that hold true for today? I'm sure there are some people for whom it must. But I've met darned few of them.

Let's take my job since I know it well. Suppose you're running a website and it is going to hit a million pages per day. (We were there a few years ago.) Suppose that you're buying $15,000 machines. (That's overkill.) 4 of them are enough to handle all of your webserving needs. We're at $60,000 and still need a load balancer and database server. So all told let's say that hardware is going to come to under $100,000. (Maybe more, maybe a lot more, if you have licensing, but let's use that as a round figure.) Let's say we are depreciating hardware over a 4 year period. So hardware is an ongoing cost of $25,000/year.

Programmers range from, let's say, $70,000 to $130,000. (Depending on the market that range could be too high, too low, or fairly realistic. The costs of different programmers depends on what you want to do.) You have, let's say, a team of 4 programmers whose combined cost is $400,000/year.

If keeping your code tightly optimized slows your team by 10%, then optimization costs you $40,000/year. That's being generous since keeping stuff optimized usually costs more than that on the maintainance side. If the result of optimization reduces your hardware needs by half then you stand to save, possibly, $12,500/year. That's being unrealistic since that load is probably not maxing out that hardware anyways, and a wise business is going to want to keep around a certain amount of excess hardware for failover and redundancy.

Spending at least $40,000/year to save at most $12,500 does not sound worthwhile to me.

Now that you've just been bored by some facts and figures, go read http://stevemcconnell.com/cctune.htm, which is what Code Complete said about optimization in its first edition. (Very good book BTW, if you have programmers who have not read it, make them read it. Now.)

Not cited in that chapter was a study that I found very interesting. I forget where I did hear about it. (At a guess, Rapid Development by Steve McConnell.) But in this study they took a series of teams and asked each one to develop the same project, but they asked them to optimize for different characteristics. One was to go for speed of development, another speed of execution, another memory use, and the last one maintainability. By and large each team's result was best on their chosen goal. However in most categories, the team that was aiming for code maintainability came in second.

What is fascinating about that is that the majority of the cost of code happens during the maintainance phase. So optimizing for maintainability is cheapest in the long run. However in addition to being cheapest in the long run, it is pretty good on every other measure that people are likely to care about!

Cheers,
Ben

[jimbob]

Let's take my job since I know it well. Suppose you're running a website and it is going to hit a million pages per day. (We were there a few years ago.) Suppose that you're buying $15,000 machines. (That's overkill.) 4 of them are enough to handle all of your webserving needs. We're at $60,000 and still need a load balancer and database server. So all told let's say that hardware is going to come to under $100,000. (Maybe more, maybe a lot more, if you have licensing, but let's use that as a round figure.) Let's say we are depreciating hardware over a 4 year period. So hardware is an ongoing cost of $25,000/year.
<SNIP>

Spending at least $40,000/year to save at most $12,500 does not sound worthwhile to me.

Now that you've just been bored by some facts and figures, go read http://stevemcconnell.com/cctune.htm, which is what Code Complete said about optimization in its first edition. (Very good book BTW, if you have programmers who have not read it, make them read it. Now.)

Not cited in that chapter was a study that I found very interesting. I forget where I did hear about it. (At a guess, Rapid Development by Steve McConnell.) But in this study they took a series of teams and asked each one to develop the same project, but they asked them to optimize for different characteristics. One was to go for speed of development, another speed of execution, another memory use, and the last one maintainability. By and large each team's result was best on their chosen goal. However in most categories, the team that was aiming for code maintainability came in second.

What is fascinating about that is that the majority of the cost of code happens during the maintainance phase. So optimizing for maintainability is cheapest in the long run. However in addition to being cheapest in the long run, it is pretty good on every other measure that people are likely to care about!

Cheers,
Ben


True for Mainfraims:

But what about apps for mobile phones? Or embedded processors? Or for RFID tags/smartcards in a few generations? Especially where computing power has to be balanced against electrical power consumption...


Spending at least $40,000/year to save at most $12,500 does not sound worthwhile to me.

What about the 4years previously (the previous computers or the one before that (gosh) say in the year 1999...

Cost of equivalent power ~2^3 times more (assuming moore's law, and 2yrs/generation) so $40K to save $100K...


Jim

[Jimbo07]

'Twas always the way. We knew this back in the 70's, when we were configuring *gasp!* minicomputers running swapping OS's (notably the first UNIX systems).



I should clarify that I wasn't drawing attention to myself by way of 'hey, lookit my profound new discovery!'

I was trying to go beyond the numbers and give people a qualitative sense of the 'feeling' of performance increase. In doing my daily Windows tasks I've noticed (gotten a distinct sense) of improved speed, no lag running multiple windows, etc.

MS's strategy has never been to write tight code (like one has to write for limited resource microcontrollers). They write bloatware and expect the hardware to be available to them. Said with a positive spin, cool new software with lots of features will be resource intensive, so if you want the next generation of 'really cool' things for computers, the hardware better keep up.

I leave it up to the reader to decide if we're seeing 'really cool'... ;)

[Diogenes]

Was this an answer to Andy's question about the number of transistors on a chip?

I'm so confused.

I actually tried to find something (anything) that actually stated the number of transistors per inch, or chip, or anything, and kept coming up with words like "Increased transistor density" or 40% increase, all nice information, but no actual transistor counts.

Anyone have a link or listing for the number transistors for say the PI, PII, PIII and P4 processors, for comparison?

I'd love to see something like that. But appears I'm too stupid to find it :crazy:

You might find this interesting..

Here is an article on Intel's soon to launch 45nm part ..

http://www.anandtech.com/cpuchipsets/showdoc.aspx?i=2915&p=3

It has 410 million transistors..

Compare that to this NVIDIA graphics processor using the 90mn process
that has 681 million. ..

http://www.anandtech.com/video/showdoc.aspx?i=2870&p=5

http://images.anandtech.com/reviews/video/NVIDIA/G80/cardshots/kentsfieldvsg80.jpg

They compare the size to Intel's 65nm process Quadcore CPU with 581 million ..

[jimbob]

I should clarify that I wasn't drawing attention to myself by way of 'hey, lookit my profound new discovery!'

I was trying to go beyond the numbers and give people a qualitative sense of the 'feeling' of performance increase. In doing my daily Windows tasks I've noticed (gotten a distinct sense) of improved speed, no lag running multiple windows, etc.

MS's strategy has never been to write tight code (like one has to write for limited resource microcontrollers). They write bloatware and expect the hardware to be available to them. Said with a positive spin, cool new software with lots of features will be resource intensive, so if you want the next generation of 'really cool' things for computers, the hardware better keep up.

I leave it up to the reader to decide if we're seeing 'really cool'... ;)

I know, it's now better than my brothers old (1987) Atari 512 ST, WISIWIG word processor (Yes that was 512 Kb)...

Mmm Bloat.

Unfortunatly, I spend a lot of time using many of the features of Word and Excel, and I havn't found a fully decent replacment. I am probably the person for whom bloatware was invented, luckily I can easily remember keyboard shortcuts...)

The default look and feel of XP was/is annoying (I wouldn't dream of getting vista before SP1...) and even then I'd postpone it as long as possible. I set mine to look like NT, *WIITHOUT* animation. (Why did they do that?)

The Unix/ linux common desktop is nice though, good for organising tasks.


Rant ends.

Jim

Edit:

Cool Moniker Jimbo

[Jimbo07]

Cool Moniker Jimbo

Likewise.

;)

[This Guy]

here we go....intel has a "fun facts on 45-nm transistors" :)


http://www.intel.com/technology/silicon/45nm_technology.htm


and here's there pdf fun facts sheet http://www.intel.com/pressroom/kits/45nm/Intel%2045nm%20Fun%20Facts_FINAL.pdf

wow.....30 million on a pin head......:jaw-dropp

Yep, that's cramming them in there! Nice info Andyandy!

[This Guy]

You might find this interesting..

Here is an article on Intel's soon to launch 45nm part ..

http://www.anandtech.com/cpuchipsets/showdoc.aspx?i=2915&p=3

It has 410 million transistors..

Compare that to this NVIDIA graphics processor using the 90mn process
that has 681 million. ..

http://www.anandtech.com/video/showdoc.aspx?i=2870&p=5

http://images.anandtech.com/reviews/video/NVIDIA/G80/cardshots/kentsfieldvsg80.jpg


They compare the size to Intel's 65nm process Quadcore CPU with 581 million ..

More cool info! Thanks!! :)

[Zep]

Who does that hold true for today? I'm sure there are some people for whom it must. But I've met darned few of them.

Let's take my job since I know it well. Suppose you're running a website and it is going to hit a million pages per day. (We were there a few years ago.) Suppose that you're buying $15,000 machines. (That's overkill.) 4 of them are enough to handle all of your webserving needs. We're at $60,000 and still need a load balancer and database server. So all told let's say that hardware is going to come to under $100,000. (Maybe more, maybe a lot more, if you have licensing, but let's use that as a round figure.) Let's say we are depreciating hardware over a 4 year period. So hardware is an ongoing cost of $25,000/year.

Programmers range from, let's say, $70,000 to $130,000. (Depending on the market that range could be too high, too low, or fairly realistic. The costs of different programmers depends on what you want to do.) You have, let's say, a team of 4 programmers whose combined cost is $400,000/year.

If keeping your code tightly optimized slows your team by 10%, then optimization costs you $40,000/year. That's being generous since keeping stuff optimized usually costs more than that on the maintainance side. If the result of optimization reduces your hardware needs by half then you stand to save, possibly, $12,500/year. That's being unrealistic since that load is probably not maxing out that hardware anyways, and a wise business is going to want to keep around a certain amount of excess hardware for failover and redundancy.

Spending at least $40,000/year to save at most $12,500 does not sound worthwhile to me.

Now that you've just been bored by some facts and figures, go read http://stevemcconnell.com/cctune.htm, which is what Code Complete said about optimization in its first edition. (Very good book BTW, if you have programmers who have not read it, make them read it. Now.)

Not cited in that chapter was a study that I found very interesting. I forget where I did hear about it. (At a guess, Rapid Development by Steve McConnell.) But in this study they took a series of teams and asked each one to develop the same project, but they asked them to optimize for different characteristics. One was to go for speed of development, another speed of execution, another memory use, and the last one maintainability. By and large each team's result was best on their chosen goal. However in most categories, the team that was aiming for code maintainability came in second.

What is fascinating about that is that the majority of the cost of code happens during the maintainance phase. So optimizing for maintainability is cheapest in the long run. However in addition to being cheapest in the long run, it is pretty good on every other measure that people are likely to care about!

Cheers,
Ben

OK, how many installations like yours would there be today? Let's guesstimate: 10,000. Multiply your costs by 10,000 = $40,000/pa * 10,000 = $400,000,000/pa.

Now the cost of just one programmer for one year (say, $150,000) to tighten up some critical bloatware code in Windows is a significant cost-efficiency ratio, is it not? In fact, a team of coders costing $1,000,000/pa would still be a great efficiency ratio.

Microsoft could do this, of course! I know some programmers there - they are GOOD at their job! But MS is a major investor in Intel and other chip makers, so it actually pays MS to force these hardware upgrades! It's what we call here a Clayton's "proprietary product" - one you have when you aren't supposed to be having one.

Incidentally, back in the day, my salary was in the region of $30,000, but a half-MEG of RAM was $500,000. So if we could save a few hundred BYTES in a 32K of program, it was definitely worth it, cost-wise.

cheers

[Ben Tilly]

True for Mainfraims:

But what about apps for mobile phones? Or embedded processors? Or for RFID tags/smartcards in a few generations? Especially where computing power has to be balanced against electrical power consumption...

Optimization versus performance is a relative scale. What is "tight code" today was "unacceptable bloat" a few years ago. For a good reason...

What about the 4years previously (the previous computers or the one before that (gosh) say in the year 1999...

...which you just showed. The result of Moore's Law is that whatever the optimal trade-off happens to be right now, in a few years will be more generous for bloat.

Of course as you change architectures, the tradeoffs change.

Cost of equivalent power ~2^3 times more (assuming moore's law, and 2yrs/generation) so $40K to save $100K...

The cost of electric power relative to programmer salary is a serious consideration for Google. It isn't a significant comparison for most other companies in the world.

Cheers,
Ben

[Ben Tilly]

OK, how many installations like yours would there be today? Let's guesstimate: 10,000. Multiply your costs by 10,000 = $40,000/pa * 10,000 = $400,000,000/pa.

I'd suspect fewer than that. But a lot that are over that size are bigger. Some a lot bigger.

Now the cost of just one programmer for one year (say, $150,000) to tighten up some critical bloatware code in Windows is a significant cost-efficiency ratio, is it not? In fact, a team of coders costing $1,000,000/pa would still be a great efficiency ratio.

Not so fast. We're not using Windows. And even if we were, the majority of our time is not spent inside of Windows code. In fact it is spent inside of language code as our languages dynamically figure out what any decent compiler would compile away. (But can't because the language is too dynamic.)

The fact is that most computers out there, most of the time, are idle. The computer is waiting for a person to do something to alleviate their boredom.

Microsoft could do this, of course! I know some programmers there - they are GOOD at their job! But MS is a major investor in Intel and other chip makers, so it actually pays MS to force these hardware upgrades! It's what we call here a Clayton's "proprietary product" - one you have when you aren't supposed to be having one.

I doubt this is a significant factor.

Incidentally, back in the day, my salary was in the region of $30,000, but a half-MEG of RAM was $500,000. So if we could save a few hundred BYTES in a 32K of program, it was definitely worth it, cost-wise.

Times have certainly changed. As I pointed out above, in many contexts today it isn't even worthwhile to use a language that presents one with the option of trying to micro-optimize.

For instance in Perl if I want to have a variable, the minimum data used is 64 bytes. (That may have changed, but it was the figure a while ago.) I want the number 1. Or a single letter. That will be 64 bytes up front before I add my data. For any operation I do I can expect it to run at least a factor of 10 slower than it would in, say, C.

Why would anyone use such a language? Well the reason is that I can develop an equivalent program several times faster in Perl than C. How much faster is a matter of controversy (and depends on the task), but it is generally agreed to be at least a factor of 5. And since programmers cost so much more than hardware does, this tradeoff is well worth it. (In this field.)

Cheers,
Ben

[Zep]

I'd suspect fewer than that. But a lot that are over that size are bigger. Some a lot bigger....remember, it's not just the USA but the whole world who uses Windows products...



Not so fast. We're not using Windows. And even if we were, the majority of our time is not spent inside of Windows code. In fact it is spent inside of language code as our languages dynamically figure out what any decent compiler would compile away. (But can't because the language is too dynamic.)

The fact is that most computers out there, most of the time, are idle. The computer is waiting for a person to do something to alleviate their boredom.True about the idleness factor - that's a product of "throwing pure CPU speed at the problem", when the REAL problem is memory real estate efficiency (i.e. paging/swapping). As mentioned above, if a program can actually be made memory-resident, there's not a lot of performance difference between most reasonably CPU models that can't be measured without microsecond stopwatches.



I doubt this is a significant factor. [Microsoft investing in Intel]It's A factor that is in the mix. MS are in the business of (1) selling their own product, and (2) pleasing their stockholders. If they can influence buyers of their primary product to take the "optional upgrade" to make it run better, they will. The side of the s/w box is where this happens. Any reasonable s/w manager knows that the "minimum spec" for Windows products is often significantly overstated - it's usually only a few key attributes that REALLY need upgrading.



Times have certainly changed. As I pointed out above, in many contexts today it isn't even worthwhile to use a language that presents one with the option of trying to micro-optimize.

For instance in Perl if I want to have a variable, the minimum data used is 64 bytes. (That may have changed, but it was the figure a while ago.) I want the number 1. Or a single letter. That will be 64 bytes up front before I add my data. For any operation I do I can expect it to run at least a factor of 10 slower than it would in, say, C.

Why would anyone use such a language? Well the reason is that I can develop an equivalent program several times faster in Perl than C. How much faster is a matter of controversy (and depends on the task), but it is generally agreed to be at least a factor of 5. And since programmers cost so much more than hardware does, this tradeoff is well worth it. (In this field.)

Cheers,
BenAgain, this was a factor in "the olden days" too. Even on mainframes we used (still use) a number of products that were quick to develop algorithms, but were heavy on the processing overhead - usually interpreters, and database query and report tools. We used them to prove the concepts, make prototypes, Q&D's, etc. But only the dumb released the resulting "programs" for "production" use, mainly because our managers were very demanding on their computing investment. After all, they didn't want their "daily sales report" take 27 hours per run! (a very personal example! ;))

When they were happy with the result, we then reproduced it in fully compiled format (COBOL, BASIC, FORTRAN, whatever). The idea was to optimise performance in the most cost-efficient way - not necessarily wring the last little bit out grunt of the box. I'm certain most of my own programs could have been somewhat more memory- and cycle-efficient, but in terms of bang-for-buck, they beat the living crap out of the prototyping tools in size and performance (e.g. the above 27-hour program came down to a 45 second runtime).

This was exactly the approach that Bell and Ritchie did with the first UNIX too. It was a really sweet little OS - tiny and efficient indeed, and still neatly extensible. Today, it would fit complete on a floppy disk, run in 256KB of memory, yet support 15+ users! On a machine with less speed than an old IBM-XT... That's how neat and sweet OS's should be, in my view! Now imagine that same OS on today's 2GHz P4 CPU with, say, 500MB RAM (i.e. very conservatively configured) - you would imagine it should support many thousands of users, or applications, wouldn't you! And yet that config today is barely able to support ONE user running the OS alone! ...see what I mean about bloatware? ;)

[Ben Tilly]

True about the idleness factor - that's a product of "throwing pure CPU speed at the problem", when the REAL problem is memory real estate efficiency (i.e. paging/swapping). As mentioned above, if a program can actually be made memory-resident, there's not a lot of performance difference between most reasonably CPU models that can't be measured without microsecond stopwatches.

Um, most PCs, most of the time, aren't blocked on memory access speed either. They are just sitting there, expectantly hoping that someone will DO something. For instance Microsoft Word has no problem keeping up with you when you're typing. Then when we finally ask it to do something, like save a file, we get irritated that we have to wait for it.

We are literally so used to having an instantaneous response that we notice when we don't get it. And don't realize that we were getting an instantaneous response all along because the computer wasn't doing anything.

I doubt this is a significant factor. [Microsoft investing in Intel]
It's A factor that is in the mix. MS are in the business of (1) selling their own product, and (2) pleasing their stockholders. If they can influence buyers of their primary product to take the "optional upgrade" to make it run better, they will. The side of the s/w box is where this happens. Any reasonable s/w manager knows that the "minimum spec" for Windows products is often significantly overstated - it's usually only a few key attributes that REALLY need upgrading.

I agree that it is in the mix, I just doubt that it is a significant factor in that mix. An example of a factor that I'd guess to be bigger is that Microsoft is aware that it is dependent on the good will of OEMs. OEMs would like excuses to drive sales of upscale products. Microsoft can do that by overstating requirements, and so does.

Times have certainly changed. As I pointed out above, in many contexts today it isn't even worthwhile to use a language that presents one with the option of trying to micro-optimize.

For instance in Perl if I want to have a variable, the minimum data used is 64 bytes. (That may have changed, but it was the figure a while ago.) I want the number 1. Or a single letter. That will be 64 bytes up front before I add my data. For any operation I do I can expect it to run at least a factor of 10 slower than it would in, say, C.

Why would anyone use such a language? Well the reason is that I can develop an equivalent program several times faster in Perl than C. How much faster is a matter of controversy (and depends on the task), but it is generally agreed to be at least a factor of 5. And since programmers cost so much more than hardware does, this tradeoff is well worth it. (In this field.)
Again, this was a factor in "the olden days" too. Even on mainframes we used (still use) a number of products that were quick to develop algorithms, but were heavy on the processing overhead - usually interpreters, and database query and report tools. We used them to prove the concepts, make prototypes, Q&D's, etc. But only the dumb released the resulting "programs" for "production" use, mainly because our managers were very demanding on their computing investment. After all, they didn't want their "daily sales report" take 27 hours per run! (a very personal example! ;))

Dumb is relative to your circumstances. I think that I've just demonstrated quite conclusively that in the right situation what was dumb is now quite reasonable.

When they were happy with the result, we then reproduced it in fully compiled format (COBOL, BASIC, FORTRAN, whatever). The idea was to optimise performance in the most cost-efficient way - not necessarily wring the last little bit out grunt of the box. I'm certain most of my own programs could have been somewhat more memory- and cycle-efficient, but in terms of bang-for-buck, they beat the living crap out of the prototyping tools in size and performance (e.g. the above 27-hour program came down to a 45 second runtime).

And in our circumstances that optimization step no longer makes sense. Doing it takes significant programmer time and will slow the addition of further features down the road. The return is negligable.

A thought-provoking essay on this topic is http://www.paulgraham.com/hundred.html. One quote I like from it is,

This isn't just something that happens with programming languages. It's a general historical trend. As technologies improve, each generation can do things that the previous generation would have considered wasteful. People thirty years ago would be astonished at how casually we make long distance phone calls. People a hundred years ago would be even more astonished that a package would one day travel from Boston to New York via Memphis.

This was exactly the approach that Bell and Ritchie did with the first UNIX too. It was a really sweet little OS - tiny and efficient indeed, and still neatly extensible. Today, it would fit complete on a floppy disk, run in 256KB of memory, yet support 15+ users! On a machine with less speed than an old IBM-XT... That's how neat and sweet OS's should be, in my view! Now imagine that same OS on today's 2GHz P4 CPU with, say, 500MB RAM (i.e. very conservatively configured) - you would imagine it should support many thousands of users, or applications, wouldn't you! And yet that config today is barely able to support ONE user running the OS alone! ...see what I mean about bloatware? ;)

I'm sure that if you actually tried to use that OS today you'd quickly become unhappy and wish that it, say, had a usable editor.

Incidentally circa 8 years ago I wound up (mostly by accident) setting up a Linux machine on a generic PC which eventually wound up with 8 simultaneous users. None were running KDE or Gnome with all the frills, but each had a private GUI (accessed through VNC) and it had no problem supporting that load. I believe that it was upgraded once since and is still in use.

Cheers,
Ben

[roger]

This was exactly the approach that Bell and Ritchie did with the first UNIX too. It was a really sweet little OS - tiny and efficient indeed, and still neatly extensible. Today, it would fit complete on a floppy disk, run in 256KB of memory, yet support 15+ users! On a machine with less speed than an old IBM-XT... That's how neat and sweet OS's should be, in my view! Now imagine that same OS on today's 2GHz P4 CPU with, say, 500MB RAM (i.e. very conservatively configured) - you would imagine it should support many thousands of users, or applications, wouldn't you! And yet that config today is barely able to support ONE user running the OS alone! ...see what I mean about bloatware? ;)
Ya, I remember those days. We had a Gould minicomputer running Unix. You'd submit your 100 line program to be compiled, go to lunch, and by God by the time you came back you only had to wait another 15 minutes for the thing to be compiled. Run it, and find out it didn't work. No problem, through some printf statements into the code using a line editor, submit the compilation again, and in an hour you'd be happily debugging. You could knock this 100 line program out in 3-4 days, easy. Of course, things would go slower when you had a lot of people logged on.

Boy, those were the days ;) Now, I'm stuck with a compiler that runs through thousands of lines of code in a minute, allows me to recompile and reinject code in a running program, while at the same time my machine downloads porn data in the background, folds proteins for Stanford, plays my music collection, gives me access to the world's literature, lets me view the world via satellite photos, discover new music using Pandora, design guitars using 3D CAD programs....

Awful, nasty stuff, that bloat...

[jimbob]

ack to the original topic: 12nm gate-length transistors are in the pipeline too... so nearly a fourfold-squared (16x) increase in transistor density.

The high -k (dielectric constant), halfnium oxide) transistors use the increased capacitance/unit area (Cox) of this oxide to provide a thicker oxide with the same Cox as for normal oxide. As it is thicker, gate leakage is less.

This is needed, because when transistor technologies are scaled-down, the gate-oxides need to get thinner too, to make the Cox rise as the technology shrinks.

Jim

[Schneibster]

<snip> go read http://stevemcconnell.com/cctune.htm, which is what Code Complete said about optimization in its first edition. (Very good book BTW, if you have programmers who have not read it, make them read it. Now.)I also recommend Rapid Development by the same author. Should also be required reading for anyone developing code.

<snip> in most categories, the team that was aiming for code maintainability came in second.

What is fascinating about that is that the majority of the cost of code happens during the maintainance phase. Hear, hear. Development without maintainability is essentially the statement that you are perfect- that you will make zero mistakes. Everybody makes mistakes. It's the nature of the business. The more maintainable your code is, the easier those mistakes will be to find and fix. Just that simple.

[Cuddles]

Moore's Law has several variations and their implications are more interesting than most people realize.

First of all outside of the IT industry nobody seems to have noticed that one of the key forms of Moore's Law has ended. It used to be that transistor counts and clock speeds both doubled quickly. But clock speeds maxed out a while ago and will not improve substantially in the forseeable future. Instead Intel and AMD are moving to putting more cores on one chip. In essence instead of getting faster CPUs, you'll be getting more CPUs instead.

The way I've always looked at Moore's Law, which I thought was the common way of looking at it, is in terms of processing power. Rather than just doubling the number of transistors or the clock speed, what is important is the operations per second. If you look at it like this it doesn't matter that clock speeds are maxed out or that we are close to limits for gate size, all that is important is that the calculating power of computers doubles every 18 months, and so far there is no sign of this trend slowing, even if some of the methods of increasing it may have reached their limits.

[jimbob]

The way I've always looked at Moore's Law, which I thought was the common way of looking at it, is in terms of processing power. Rather than just doubling the number of transistors or the clock speed, what is important is the operations per second. If you look at it like this it doesn't matter that clock speeds are maxed out or that we are close to limits for gate size, all that is important is that the calculating power of computers doubles every 18 months, and so far there is no sign of this trend slowing, even if some of the methods of increasing it may have reached their limits.


The power is increasing *because* it has been possible to shrink the basic CMOS technology. It is *still* shrinking, but the fundamental roadblocks are ahead.

Of course there are some interesting potential approaches to improve processing power after that, but the fundamental nature of the technology will have to change from CMOS logic.

Molecular computing/quantum photonic flip-flops etc...

They could render PGP encryption pointless.

It is a fair bet that anything that is compatible with Silicon CMOS will have a huge advantage, due to industrial inertia:

I had a lecturer who said of Gallium Arsenide "if God had wanted us to make GaAs chips, he wouldn't have made beaches out of sand."

Silicon is also nice and easy to handle... Not like some of the other compounds.

Jim