Macros

[Paul C. Anagnostopoulos]

Ah, macros. Some people love 'em, some people hate 'em. I think they're one of the greatest things since sliced pajamas, assuming the macro facility has some decent power. This rules out the stoopid #define facility of C++ et al. Java didn't even bother with them.

IBM Assembler G and H: Now there was a macro facility. I used it as the model for the macro facility in Gossip. Here is the quasi-BNF for the macro facility, just so you can get a bit of an idea.


macro-def: #MACRO {id | bang-name} = group ;
parameter-list group #ENDMACRO/#MEND

bang-name: !x (x any character)

parameter-list: ( [param-decl-list] [, rest-param] )

param-decl-list: {id [:= balanced-token-seq]} >< ,

rest-param: REST id

if-statement: #IF expression #THEN/#DO group
elsif-clause* [else-clause] #ENDIF/#FI

elsif-clause: #ELSEIF/#ELSIF/#ELIF expression #THEN/#DO group

else-clause: #ELSE group

set-statement: #[O]SET expression ;

loop-statement: #DO group #END

over-statement: #FOREACH [foreach-mod] id [foreach-mod]
IN [foreach-mod] expression #DO
group #END

exit-statement: #EXIT ;

exitif-statement: #EXITIF expr ;

terminate-statement #TERMINATE ;

message-statement: #REMARK expression ;
#WARNING
#ERROR

trace-statement: #TRACE id [: expression >< ,] ;

ignore-construct: #IGNORE group #ENDIGNORE

substitutor: #? {id | ( expression )}

#. id token from string, string/id token
#= token(s) from integer, real, string, token, token sequence, NIL
#" string token from string, id/keyword/operator token

macro-call: !( arg-list )
![ arg-list ]
!{ arg-list }
!x token (if macro takes 1 non-rest parameter)
name ( arg-list )
name [ arg-list ]
name { arg-list }
name arg-list {; | DO}

[Iconoclast]

Originally posted by Paul C. Anagnostopoulos
Ah, macros. Some people love 'em, some people hate 'em.In my experience, the people who "love 'em" are the people who write them, the people who "hate 'em" are everyone else in the team who have to deal with them.

Originally posted by Paul C. Anagnostopoulos
Java didn't even bother with them.I don't know anything about Java, but macros were omitted from C# because of the huge potential for abuse. The problem with macros (like regexes) is that they can be much easier to write than to read. Now, there certainly ARE cases where a function cannot be used in place of a macro, but they're fairly rare, I try to avoid using them unless absolutely necessary. In C++ at least, when a developer has to maintain a module that contains macros that he didn't write, every time he comes across one in the code he has to dig up the header file to find the macro. For any complex macro, more often than not it has been written as an indecipherable jumble and so the developer has to copy the macro code to notepad and manually expanded it in order to understand exactly what it does. Perhaps I've only ever had to maintain macros written by "hacks".

Even worse, some developers use macros for flow control (http://blogs.msdn.com/oldnewthing/archive/2005/01/06/347666.aspx). This is incredibly dangerous, I don't see how a developer could reasonably maintain code that contains macros than control program flow logic.

[Paul C. Anagnostopoulos]

Ah, all these people are just wimps. What's the difference between looking up a macro definition and looking up a function definition? Either you know what it does or you don't, and if you don't then either you need to know or you don't.

I certainly agree that people can write inscrutable macros. There may be a tendency not to comment macros. I also agree that overly complex flow-of-control macros are probably a mistake. However, I think the following is clearer and less error prone:

foreachchild i, node do {
blah blah
}

than this:

for i := 1; i <= node.nchild; ++i do {
blah blah
}

Regarding patterns and strings, the language should support escape sequence so you can comment them. Gossip has the \/ escape sequence, which ignores the rest of the line and following whitespace.

Of course, I'm the only one who ever reads any of my own Gossip code, so this is all easy for me to say.

~~ Paul

[Paul C. Anagnostopoulos]

It should also be noted that many complaints are based on the C++ macro facility, which is a busted-ass piece of junk.

~~ Paul

[Stimpson J. Cat]

I'm having a hard time understanding why anybody would want to use macros in C++ at all. I can understand using them in C, but in C++ there just doesn't seem to be any point.

The only time I use the preprocessor at all is when I am trying to make code work on multiple platforms, or when I want to add extra error checking that is only present in debug builds.

By far the nastiest C macro monstrosity I have ever seen is in the fftw3 header file (FFTW is a C library for doing FFTs). They declare all of their prototypes and enumerators in one huge macro. That way they can automatically implement name-mangling and generate them for float, double, and long double. Basically they are using the preprocessor to do templates.

The macro is 160 lines of code. Needless to say, the code is pretty much unreadable.


Dr. Stupid

[Paul C. Anagnostopoulos]

160 lines of well-documented code, I'm sure. Holy cow.

I mostly use macros for shorthands, which is probably a holdover from my assembler days and my years with Common Lisp. For example, I write a lot of tokenizers and parsers. The tokenizer would be full of things like this:

elsif ch == "." then { token &:= ch;
ch := next_source_char();
state := statePOINT; }

over and over again. It's so verbose it's difficult to read, gets spread out, doesn't fit in my editor window, etc. So a couple of judicious macros turns it into:

elsif ch == "." then { token &:= ch; next; !>POINT; }


The other great use for macros is generating complex data structure initializations from simpler table-like specifications. If the initializations are too complex, I invent a little language for them, code them in a separate file, and write a compiler for the little language. But if they are only moderately tricky, I write macros.

I agree that it is probably asking for trouble using macros for control structures much more complicated than my foreachchild example above.

~~ Paul

[Iconoclast]

Originally posted by Paul C. Anagnostopoulos
Ah, all these people are just wimps.If you're a one-man team who writes and maintains his own code and nobody else has to look at it, then sure, knock yourself out, use all the "tricky" constructs you like in your code. For group projects, or where someone other than the original author has to maintain the code, you must realize that every little trick you put into your code will slow down the next poor schmuck who has to look at it.

Originally posted by Paul C. Anagnostopoulos
What's the difference between looking up a macro definition and looking up a function definition?For one thing, you can't step through the source of a macro in your debugger since it doesn't exist after your precompiler touches it. I'm talking about C++ here, I don't know how macros in other languages such as yours are implemented.

Originally posted by Paul C. Anagnostopoulos
However, I think the following is clearer and less error prone:

foreachchild i, node do {
blah blah
}

than this:

for i := 1; i <= node.nchild; ++i do {
blah blah
}
But, you've "invented" a new construct here, If I was reading this code I'd have to dig around and find out what "foreachchild" actually did before I could continue. I need to find out if it can throw exceptions, I need to find out what it does if it comes across a null child node, and whether or not it blows up if the top level node is null or whether it has a check for this condition. Sure, I could make assumptions about all these things, but that would not be a rigorous way to ensure the code does what it's supposed to. Regarding the second example, it may be more verbose, but it will necessarily be understandable by any developer who has learned whatever language it is written in.

Originally posted by Paul C. Anagnostopoulos
Of course, I'm the only one who ever reads any of my own Gossip code, so this is all easy for me to say.Like I said, if you're the only one who has to deal with your code, you can do whatever you like, but overuse of macros in commercial source code is a real problem.

[Iconoclast]

Originally posted by Stimpson J. Cat
I'm having a hard time understanding why anybody would want to use macros in C++ at all.There are certain things you can do with macro code that you can't do with C++, and it's in regard to collapsing often repeated code blocks that cannot be pushed into C++ methods.

Consider the following, my team once used code like this in a COM+ business layer in a very large commercial application, it's a long time since I used ATL, so the syntax is probably all wrong:

STDMETHODIMP MyMethod(IUnknown** ppUnk, double dValue)
{
BEGIN_METHOD

.... do stuff

END_METHOD
}


Where BEGIN_METHOD was defined as something like:

try{

...and END_METHOD was defined as:

}
catch(ExceptionType1)
{
... do exception logging and cleanup
}
catch(ExceptionType2)
{
... do different logging and cleanup
}


These macros obviously can't be implemented as functions since one has a "try" and the other has all the "catch" handlers. In this case, we carefully documented these macros to ensure all the develoipers knew exactly was going on (there was actually a lot more code than I've shown), and these constructs meant the methods we wrote included only the "interesting" code and at the same time ensured that any exceptions (and tracing from memory) were always handled correctly.

Still, I ain't no macro fan.

[LW]

Originally posted by Paul C. Anagnostopoulos
[B For example, I write a lot of tokenizers and parsers.[/B]

Why?

What's wrong with lex, yacc, and n+1 other lexer and parser generators?

[Paul C. Anagnostopoulos]

Iconoclast said:
If you're a one-man team who writes and maintains his own code and nobody else has to look at it, then sure, knock yourself out, use all the "tricky" constructs you like in your code. For group projects, or where someone other than the original author has to maintain the code, you must realize that every little trick you put into your code will slow down the next poor schmuck who has to look at it.
You keep using the word tricky. I agree that we should avoid tricky stuff. I don't think macros are tricky by definition, and certainly no more tricky than a bunch of other things you could do.

For one thing, you can't step through the source of a macro in your debugger since it doesn't exist after your precompiler touches it. I'm talking about C++ here, I don't know how macros in other languages such as yours are implemented.
Oops. Well, that certainly dates me. I'm not one for stepping through source code in debuggers.

But, you've "invented" a new construct here, If I was reading this code I'd have to dig around and find out what "foreachchild" actually did before I could continue. I need to find out if it can throw exceptions, I need to find out what it does if it comes across a null child node, and whether or not it blows up if the top level node is null or whether it has a check for this condition. Sure, I could make assumptions about all these things, but that would not be a rigorous way to ensure the code does what it's supposed to. Regarding the second example, it may be more verbose, but it will necessarily be understandable by any developer who has learned whatever language it is written in.
You need to find out if a function you call can throw exceptions. You need to find out how an iterator behaves when there are no children or one is null. (Perhaps you're not a big fan of iterators either.)

Like I said, if you're the only one who has to deal with your code, you can do whatever you like, but overuse of macros in commercial source code is a real problem.
I'm sure it is. I tend to use a few macros in each module, and they are defined at the top along with the data structures.

~~ Paul

[Paul C. Anagnostopoulos]

Iconoclast said:
Still, I ain't no macro fan.
Yet without them you would have had to rely on everyone catching the correct exceptions.

I suspect part of the secret to rational macro usage is good documentation. I think macros tend to be slapped together and then ignored.

~~ Paul

[Paul C. Anagnostopoulos]

LW said:
Why?

What's wrong with lex, yacc, and n+1 other lexer and parser generators?
I'm not writing in C. When I was, those packages were not some of my favorite things. Anyway, I enjoy writing tokenizers and parsers.

~~ Paul

[Iconoclast]

Originally posted by Paul C. Anagnostopoulos
You keep using the word tricky. I agree that we should avoid tricky stuff. I don't think macros are tricky by definition, and certainly no more tricky than a bunch of other things you could do.I not saying that this applies to you, in fact I'm sure it doesn't, but quite often I've had the frustration of dealing with developers who are "geniuses", they have the attitude that if the rest of the team can't understand their code it's because they're all idiots. This is a silly rationalization. I bet you could write a piece of code that I can't understand, and I'm pretty sure I could write code that you can't understand -- in fact there's a code obfuscation contest every year -- but writing such code does not make a great developer. A great developer writes code that ANY IDIOT can understand, even me.

Hey, I just thought of an idea for a new thread!

Originally posted by Paul C. Anagnostopoulos
I'm not one for stepping through source code in debuggers.In my experience interactive debuggers are the fastest way to track down bugs in (unoptimised) code. That's why they were invented.

Originally posted by Paul C. Anagnostopoulos
You need to find out if a function you call can throw exceptions. You need to find out how an iterator behaves when there are no children or one is null.True enough. But in your second example there is no ambiguity, I only need to know the rules of the underlying language to know what's going on.

Originally posted by Paul C. Anagnostopoulos
Perhaps you're not a big fan of iterators either.Hey man, I love iterators. They're great when you need to iterate across a collection to perform operations that aren't a function of the position of the elements within the collection.

Jeez, that sounds like a bad textbook, how about I try English:

If I need to (say) put the values of a collection into a speadsheet, I'll use a "for" loop, since I know I'll need to know each element's index in order to write it into the correct cell. Any time I don't need to use the element's index inside the loop, I'll use an iterator.

Originally posted by Paul C. Anagnostopoulos
Yet without... [macros] ...you would have had to rely on everyone catching the correct exceptions.Quite true. I'm not one to place absolutes on whether some construct or another is "bad" or "good", I realize that in the case I cited that macros were the neatest solution to the problem. I would still assert that your example needlessly complicated the problem, even though it leads to more compact source.

Originally posted by Paul C. Anagnostopoulos
I suspect part of the secret to rational macro usage is good documentation. I think macros tend to be slapped together and then ignored.Agreed.

[Paul C. Anagnostopoulos]

Icono said:
I not saying that this applies to you, in fact I'm sure it doesn't, but quite often I've had the frustration of dealing with developers who are "geniuses", they have the attitude that if the rest of the team can't understand their code it's because they're all idiots. This is a silly rationalization. I bet you could write a piece of code that I can't understand, and I'm pretty sure I could write code that you can't understand -- in fact there's a code obfuscation contest every year -- but writing such code does not make a great developer. A great developer writes code that ANY IDIOT can understand, even me.
I'm all for code any idiot can understand. Well, almost any idiot.

In my experience interactive debuggers are the fastest way to track down bugs in (unoptimised) code. That's why they were invented.
Perhaps. I've been around too long, I guess. The primary requirement for debugging is the ability to make hypotheses about what the problem might be and then test those hypotheses, one at a time. I find people just flail about until the program behaves more or less like it should.

I tend to use the old trace statement technique. I've certainly used various interactive debuggers, old style and new. They were okay.

If I need to (say) put the values of a collection into a speadsheet, I'll use a "for" loop, since I know I'll need to know each element's index in order to write it into the correct cell. Any time I don't need to use the element's index inside the loop, I'll use an iterator.
Gossip isn't object-oriented, but it has two variants of the foreach statement:

foreach i in collection do ... When you need the index.

foreach i, item in collection do ... When you need both the index and item.

Perhaps I should consider:

foreach *, item in collection do ... When you only need the item.


Quite true. I'm not one to place absolutes on whether some construct or another is "bad" or "good", I realize that in the case I cited that macros were the neatest solution to the problem. I would still assert that your example needlessly complicated the problem, even though it leads to more compact source.
In the case of tokenizers, the size is the complexity, because it's the same code repeated over and over. The definitions for the next and !> macros are trivial, appear just above the tokenizer FSA, and make the FSA more readable. That's my story, anyway.

~~ Paul

[Thumbo]

Originally posted by Paul C. Anagnostopoulos

The primary requirement for debugging is the ability to make hypotheses about what the problem might be and then test those hypotheses, one at a time. I find people just flail about until the program behaves more or less like it should.


Absolutely agree - but why isn't this taught in computer science courses?

I've tried drilling it into my team, but I still get the "the bug went away when I added debug, so I put a sleep in and that fixes the bug."

How can one persuade them that the bug is not fixed until you understand why it happened in the first place?

I wonder if age is a factor? I went through university before computer science degrees were common: my own background is in Physics. I don't remember a whole lot of it, due to lack of use, but the experimental method was drummed into us - and its just as applicable to exploring the behaviour of a complex and not fully understood man-made system as it is the natural world.

[Wudang]

If I can only find a bug in my program using an interactive debugger how does the person running my code find out what's gone wrong?
Carmine Cannatello's "Advanced Assembler Language and MVS Interfaces" has some good stuff on macros. Of course he has the advantage that assembler is self-documenting.
My favourite macros
MODESET KEY=ZERO,MODE=SUP
MGCRE MF=(E,LAREA),TEXT=(R2),
CONSNAME=MASTER

[Iconoclast]

Originally posted by Wudang
If I can only find a bug in my program using an interactive debugger how does the person running my code find out what's gone wrong?I don't think anyone in this thread has claimed that they can only find bugs using an interactive debugger.

For the case of diagnosing errors when running in the production environment, any good enterprise software should have extensive logging and exception reporting facilities built into it's architecture. The applications I build certainly do, whenever a user gets an exception raised, I get an automated email and can usually track down the problem before they have a chance to phone me.

[Iconoclast]

Originally posted by Paul C. Anagnostopoulos
Perhaps. I've been around too long, I guess. The primary requirement for debugging is the ability to make hypotheses about what the problem might be and then test those hypotheses, one at a time. I find people just flail about until the program behaves more or less like it should.That has little to do with the method of debugging used, it's an issue of bad debugging techniques.

Before modern debuggers existed, some people would STILL trace out every variable they could find and then look for any "weird" values. Interactive debuggers simply let you view the state of your application faster and with much less effort, it's silly to say that they are somehow "bad" because of this ability.

Modern OO apps tend to hold a lot of state, the ability to show the entire state of an object with two mouse clicks is an asset, not a liability, but logically thinking through a problem is STILL a basic requirement if one wishes to track down and fix bugs in a timely manner.

[Paul C. Anagnostopoulos]

Iconoclast said:
For the case of diagnosing errors when running in the production environment, any good enterprise software should have extensive logging and exception reporting facilities built into it's architecture. The applications I build certainly do, whenever a user gets an exception raised, I get an automated email and can usually track down the problem before they have a chance to phone me.
Now you're talking!

That has little to do with the method of debugging used, it's an issue of bad debugging techniques.
Agree, I didn't mean to say otherwise. I was trying to say that the methodology is more important than the tool.

~~ Paul