Testing – particularly automated unit and acceptance testing – drives me a bit mad.
When we want to assert something about data flow, we make a bunch of assertions about particular, peculiar data values moving back and forth. When we want to assert that you can make a post from either the admin or the public-facing side and have it appear on both sides, we test those four facts independently, that a post can be both written and seen on both sides; or we test two pairs of operations.
What we mean (or at least what I mean), and try to get at with all this, is that conceptually there is a single entity with two read operations and two write operations available.
What would it take to have an application and a test framework that lets us say that?
While I am here, please allow me to complain about mocks. I know exactly how often I care about a particular method being called, and it is precisely never. What I care about is that whenever a particular thing happens, a bit of data crossing an abstraction boundary or a computation being performed, some method acts in a particular way. If you isolate the action of the method from the boundary or the computation, then you isolate it from its meaning and demolish the value of the test.
And I hate maintaining tests that aren’t valuable.
Fine Shambles 
The last part sounds like an argument against unit testing (only do integration testing). I’ve recently made the inverse discovery. My Mars (compiler) test suite is purely integration tests: I can only make tests by supplying an input program and seeing what the output is. It turns out it makes it really hard for me to test individual components of the compiler. If I vaguely suspect a (compiler) function has a bug and desperately want to supply a certain input, I often find it very hard to construct a program that will cause that (compiler) function to receive the certain input I require. Unit tests would be quite helpful.
Also, I just recently did some mock testing with App Engine. As you saw on Stack Overflow, I wanted to have a fake User object, because I needed to test exactly how the program reacts when differently-classed users are logged in, and the only way I could do that was to mock the User object, and allow the test cases to individually load fake User objects. (And I got it working.) Isn’t that cool?
I’m trying to make an argument against doing unit testing in some cases. When the “unit” being tested sits between two pieces, and has relatively little importance apart from the connection between the two pieces, then testing it as a unit captures only some tiny fraction of the stuff that you actually care about. I find it a bit difficult to talk about the compiler case that you cite, because almost of the cases I’ve actually encountered do very little actual computation.
Consider a system with pieces A, B, C, such that data flows from A through B to C. I don’t want to write that A calls B, that B calls C, that B does X when called with Y, and B does X’ when called with Y’. I want to assert that data flows from A to B to C, and that if the data that flows from A to B has these properties, then certain things will result.
What such assertions would look like, I still do not know.
Sounds reasonable. I don’t want a unit test on every function. But I do want unit tests on certain hairy functions that I can’t easily get at with an integration test. I suppose I like integration tests for the full functionality of the system, and unit tests where it makes sense.
For the compiler case, consider a (fictional) compiler that tracks whether or not a variable is used before it is assigned. Deep in the bowels of the compiler there is a function updateDefStateStmt that takes a DefState (state of all variables) and and a statement, and updates the DefState and possibly raises an exception, if a variable is assigned but not yet defined. Now I can think of a whole bunch of hairy corner cases and situations where updateDefStateStmt might go wrong, but trying to come up with a source program which would cause each of those situations would be a) tiresome, having to think of a complete program for each case, and b) sometimes impossible — there may be certain valid inputs to updateDefStateStmt which really should work, even though it is not currently possible for them ever to arise in real usage. Thus, writing some unit tests for updateDefStateStmt would be ideal.
Currently on Mars, I can’t do this. I have to think of a full program for every case.
Also, while I am here, Python’s doctests are one honking great idea — let’s do more of those!