“Did MS consider that cache awareness for programmers in multicore development?”

…and he answered:

“The average developer is not capable of handling that kind of level of detail. … Most developers are that ignorant. Welcome to the real world.”

To me, this explains a lot. It explains why .NET looks like it does, and to clarify what I mean by that, let me simply copy in extracts from what I had to say about it in a private forum just weeks ago. In what follows, the italics are brief extracts of comments from others. The rest is my own text. It’s not always in a totally logical order and it starts out in midflight, but it’s a synthesis of a longish thread on a security related forum.

Note: I didn’t ask any of my correspondents for permission to quote, but the quotes are so limited that I consider them “fair use”. If you think I crossed a line here, beat me into submission and I’ll never do it again.

- – - here we go – - -

Even Java, Ada, and COBOL  do not inherently encourage “good programming style”.

Well, I wasn’t thinking of which language “encourages” good programming practices. IMNSHO, none really does. That’s an idea that isn’t even viable once you get into programming for real. The idea of “encouragement” from a language standpoint is, let me be blunt, something only wannabe programmers could think is relevant.

Once you really use a language and its platform, one is not bothered in the least about what the language or platform “encourages”, but what the language or platform *allows*, and that’s an entirely different kettle of fish. And this is where .NET falls down severely.

In .NET you can do reflection. Up to a point, and then not. You can do generics/templates. Up to a point, then it comes to a screeching halt. You can do aspects, but only to a very small point. You can do dynamic scripts, but only to a severely complicated and fairly worthless point. And on and on it goes. For every good abstraction technology they implement, you soon discover it’s just a faked out front that can’t scale up to something really useful.

So, like MS Basic before it, .NET languages and the platform have cheap selling points, leading you down a path of instant gratification and then you hit a wall. But it’s actually much worse than good ol’ MS Basic was, since MS Basic was actually pretty close to what could be done back then, which wasn’t much. And you hit the wall soon enough to realize you should take another path. .NET, on the other hand, limits you from exploiting what currently very well can be done with computers, and it let’s you invest considerable energy and time in the platform before you get down to the bare metal behind it, only to discover it’s a rusty shell of improvised junk.

Let me put this in yet another way. Limiting languages to what is considered “safe” by non-programmers may limit the damage relative amateurs can do with these languages. But if this limitation also means you limit the *depth* of the language, like .NET does, then you actually put a ceiling on the level of abstraction than can be reached in larger systems built by people who actually know what they’re doing. Meaning that you exchange junk on a small and local scale for architectural and design junk on an epic scale.

With the reality of rapid time to market/deployment and a realistic assessment of the skills many programmers bring to the table, limited is what is needed quite often.

Yes, but what… um. Let me restart. It’s not a “yes, but” thing.

No, that’s wrong. Because it let’s you build small systems quicker and safer, that’s true, but small systems almost always become large systems. And these large systems almost always become monsters, due to the limitation that was built into them from the start. IOW, the limitation you talk about will practically always result in severe problems a bit down the road. It rewards shortsighted development.

Maybe, just maybe, limited and “safe” languages will reduce stupid errors in small systems by unskilled programmers more than they will cause deep errors in architecture of large systems (written by skilled or unskilled programmers, since skill doesn’t bring much to the table if the system won’t allow advanced techniques), but do we know that to be true? Has anyone tried to find out? As long as we don’t know that, you buy into “safe” languages on no better basis than what you do when you choose one shampoo over another.

- – -

Let me take a real life example to show what I mean, and why I currently detest .NET so much. (Maybe, just maybe, someone on the forum then tells me the one feature I missed in .NET and everyone gets to laugh at me, but I’m willing to risk that. I’m actually willing to *welcome* that, since it would be such a relief.)

When doing an app with grids and details, like 90% of everything out there, you want to let the user change details, add lines, delete lines, whatever, and then hit “Save” to save it all to the database, or “Revert” to go back to the original state. You also want the system to ask “Do you want to save first?” when you try to leave without saving changes. Right?

Ok, there are several ways of doing this. The “right” way would be to have the objects in memory versioned. You should be able to ask any object “did you change?” and “what did you change?”, and tell the object to roll back to a previous state or commit to the current state and start recording changes from there from now on.

You can’t do that in .NET, there’s no such feature. So, you say, what else is new, let’s fix that. With, um, AOP, which sounds just about right. Looking for solutions on the net shows nothing but spurious outbursts of unprintable language and gnashing of teeth. So forget it.

A mix-in, then? Um, no, .NET doesn’t support mix-ins, that’s only if you’ve got multiple inheritance.

Ah, let the objects deep-copy themselves to save state. Um, no, .NET has no reliable or complete deep-copy. And even if it had, it would copy too much of dependent object trees.

Then, let’s try good ol’ raw bitblitting into memalloced save spaces and calculate our own offsets (and make the company CISSP, if there is one, turn blue in the face)? Um, no, such “unsafe techniques” aren’t allowed in .NET.

Let’s inject the dispatch table with a pre- and post-function to record changes (key/value observing or AOP, depending on taste)? Nope, can’t do. Unsafe.

Finally, if you still haven’t dumped the whole idea and started a bicycle shop instead, you may have worked your ass off to get a kind of reflection function lib going that can copy, compare, rollback your objects, if your objects conform to a fairly limited feature set and implement some clumsy attributes. That’s what I did, and I keep debugging this thing long past the time when it should have been clean. I’m also pretty sure it’s slow, but I don’t care anymore.

Or, you do what everyone else does, and that is riddling your code with checks if anything changed, and saving old values here and there when you think they may need to roll back. So if there are 15 ways of leaving that screen, you’ve got 15 different places you need to check if you need to reload or save any of maybe 20 different objects in your app. Leaving the support desk a never ending source of joy and happiness for years to come as customers find new and inventive ways of leaving half edited objects in memory.

You know what? If this had been C++, I would have done a real down to the metal hack of the dispatch vtable mechanism maybe, and gotten a fast solution working that could be used anywhere and forever to solve this problem. It would be “dangerous” but once debugged it would last forever.

What you see now in .NET is much less dangerous solutions, but they have to be done over and over again and everytime they risk new bugs, each of which is much harder to find than any bug I would have in my “unsafe” solution.

BTW, even my halfassed solution won’t work if you compile .NET apps in “secure” mode (or whatever it’s called) since reflection can’t be used then. (Meaning an even buggier solution with hundreds of times more code for the same thing…) And you know what, it’s actually *much more* difficult to get the 15 * 20 simple-code solution debugged than the one advanced vtable injection technique debugged.

So, that’s what “safe” languages deliver even for pedestrian apps. Hurrah.

- – -

Not sure I’d let most developers any where near C or C++.

Yes, but you also lose the opportunity to get really good scalable apps. I would suggest finding a few “real developers” instead of a load of “most developers”. Quantity can’t compensate lack of quality.

As for .Net, you could always mix managed and unmanaged code and do what you need to do in C++ if you’ve got the guys with the skills to do it right.

If that would let me attack the basic dispatch mechanism in .NET and have it install itself into all the objects in memory that I pointed out to it, with for instance attributes, I’d do it. But, AFAIK, you can’t do that. Then you need to switch over to C++ unmanaged for the entire project. Which, if we look back, was my exact argument. You *can’t* do scalable and safe programming in managed .NET even if you’re able to.

My argument is this: if you cripple a language, you’ve exchanged a few “dangerous” and qualified pieces of code against a sh..load of slightly dubious and simple pieces of code. And of the two ills, I am convinced the latter is the worst.

PS: I’m still waiting for someone to tell me I should’ve used disconnected datasets. Here I am, fully loaded for war and nobody shows up?

- – -

I’ve found it surprisingly practical on the scales I normally work with. I certainly concur that it’s not what you might call “large enterprise ready”, but then what really is?

BTW, couldn’t we define a “large system” as one in which none of the builders (are really able to) know what everything is doing?

I’ll take those two as the same question.

It *is* surprisingly practical for small scale stuff, or as the technical term goes: quick and dirty. But what is a “large system”? I’d say most useful systems are “large systems”, since the builder, even if he’s alone, can’t keep all parts of the system in his head all the time. I don’t think there are any, or more than very few, really “small” and useful systems out there, according to this definition.

A good programming system allows you to write “straight code”, or quick-and-dirty, as you start out and as soon as you see a pattern in your code, you are able to replace that pattern with a single implementation of an abstraction somewhere. All on the basis of DRY (Don’t Repeat Yourself). You *have* to be able to do this, else you get inheritance-by-editor, that is repeated and almost identical code in multiple places. Which is the most sure-fire way of making it a unmaintainable slag heap in no time.

For simple code sequences, we have functions. Even C# can do that. For structures we have objects. C# can, yay. For behaviours, we have templates aka generics. C# does that better than C++ in the “cute and nice to have” ways, but doesn’t do it at all in the “OMG-what’s-this- weird-code-but-it-saves-the-project” ways. It’s just cute and tantalizingly close to being real-world useful. Aspect oriented programming (AOP) or Key-value observing (KVO): C# is totally braindead. Same for run-time object extensions. Same for object delegation or runtime discovery of implementation (cfr respondsToSelector in Objective-C). None of these things are even there in any form. Even swapping out one DLL for another in runtime is painful in the extreme.

A highly abstract way to view this is to visualize a development effort curve that starts out linearly, then goes exponentially up. You stop there, grab the language by the neck and raise your code an abstraction level. You drop back to linear for a while, then you go exponential again, then you raise the code an abstraction level, etc.

With .NET, you can do this for a couple of times, then you either run out of abstraction levels or they don’t return you to a linear curve, but to another exponential curve (IOW, whatever you do, C# bites back). With C++ I’ve never run out of abstraction levels, they go on and on, right into user domain territory to an amazing degree. I suspect Objective-C won’t run out of them either. Languages like PHP don’t have them at all, they’re more or less at the level of MS Basic way back when. Almost. Which in itself is a kind of honesty, since PHP doesn’t even pretend to be a “real” language in that sense. It’s quick- and-dirty elevated to a religion.

Even though I can’t prove it, I strongly suspect that there are no simple projects really suitable for .NET. Almost all projects start out being the right scale, but almost all of them ultimately (or even “quickly”) escalate beyond the level where .NET is a good fit, but then they’re stuck on that platform. And it is in this phase of its lifecycle that we see the problems with unmaintainability and bugs, not while they’re still small.

So, in conclusion, I strongly doubt that “safe” programming systems bring us that safety. I think it’s the other way around. The problems we see due to poor use of sharp knives (horrible C++ code, for instance) can’t be solved by switching languages, but should be solved by switching or training programmers and fixing project management issues.

If you don’t have the project management and the developers you need to write solid, maintainable, and safe code with a real language, switching to “safe” languages won’t help you. I think that is what I’m trying to say.

- – -

A well-designed module will be maintainable. “Take the time to do the design before coding” beats the snot out of “pick the write language” when it comes to making the source readable and maintainable.

Uh, no. And yes. And no again.

The problem is that if you design your stuff first, the language has to be able to express that design. Unless you keep the design trivial enough to be expressed by any old (new) language.

For instance, if your design has the element “Restore this object tree to last savepoint” and your language has a feature that can be used to achieve that, you may get away with a handful of lines of code that clearly and succinctly express what you’re doing even without comments or docs, and that can be tacked onto your objects as mix-ins, categories, templates, reflection, or whatever.

If your language does not support the necessary operations and declarations, you’ll end up with stereotypical code that needs to be manually replicated into each and every darn class you’re building and that is specific for the very project you do them in, hindering reuse across projects. You end up with hundreds or thousands of lines of code spread all over, hard to understand, even harder to find when you need to find it and a real f…up to debug.

You can’t get around that if the language is too limited. Or “safe” as they call them nowadays.

I have seen readable structured code in assembler and incomprehensible spaghetti in Modula-2

Assembler is much less limited than C# in this respect, so that isn’t so strange. It starts out at a very low abstraction level, but can reach very, very high. I’d say assembler is more or less linear in development complexity, but the problem is that the line, however straight, is very long.

Also, if you’ve already concluded that assembler can be well structured, then you have to admit C can as well. Same thing, almost. Extend at will to C++.

Of course it can!  Any language can!  That’s my point….

Hah! OMG… so tell me, what’s wrong with MS Basic, then? Or MSDOS batch language?

Do an undo/redo manager for MS Basic, or even (let’s be generous here) C#, in an average accounting app. Then in Objective-C. Then compare for “well structured”. I’d guarantee you that the C# undo/redo manager will be unreadable, plastered all over the project up to the rafters, a neverending source of joyous bugs and a strongly contributing cause to the death of the product just a few years hence. Not so with Objective-C. (The undo/redo manager is already part of the libraries, but even if it wasn’t, it could be built by the developer in a very reasonable time. Not so for C#.)

And no amount of design will change that. C# simply doesn’t have the features you need and is brimming with the “safety” to stop you from getting it done. Thanks to the drive for “safer languages”, I have to add.

Isn’t the CISSP-relevant point here that your redo manager, or any comparable object, necessarily constitutes a locus of increased risk? Seems to me that might be why it’s hard to implement in such a language.

Yes, maybe, which points out why the CISSP-relevant point is nuts. Somebody decides that the undo/redo manager (or something equivalent) is dangerous, so let’s not do that. Um, so what do we do when we don’t have an undo/redo? We fake it in each of the 50 or 100 classes that need it and the problem is now orders of magnitude larger. Or, more likely, we prohibit undo/redo entirely, making the use of the app a bloody nightmare, causing the actual business to suffer. And if the bloody user makes an error, he can’t correct it, but then it’s his fault, right?

In any case, this method of suppressing any patterns that carry risk only moves the problem elsewhere, usually making it much worse at the same time.

Pretty much all languages these days do the real work with library calls. The accretion of platforms slowly renders specific languages less relevant, as implementations increasingly consist of calls to external libraries and objects held outside the language’s environment. So-called “safe” languages attempt to fulfill their promise by limiting external calls or, in the .Net case, opening holes to “unmanaged code”.

I intentionally took an example of something that can’t be done by libraries. You can’t equip classes or objects with runtime state monitoring by library calls if there is no dynamic injection or intercept functionality in the language. There is no equivalence between language functionality and library functionality; those cover disparate areas of functionality. Libraries have in absolutely no way taken over from languages, but libraries have grown much faster than languages, so it may seem so if you just compare sizes.

This is the dichotomy of Java: it lives in a sandbox that makes it very safe, leaving you with the problem of getting things in and out of the sandbox.

As taken right out of the marketing book. It’s BS. If it was as easy as prohibiting dangerous behaviour and then danger went away, Java would have solved world hunger by now.

Remember the monkey dance? Ballmer yelling “Developers, developers, developers!”, while jumping around like a neurally defective and sweating profusely (one could be excused for suspecting some cholinergic poison, but he lived through it, so that is not the answer). Right. I mean, he’s right. Developers is what makes or breaks a platform, but now he’s losing them, so he really has no reason to celebrate.

When Apple designed the iPhone, they could have created a special development system and language for it, but even though it may have been easier, they didn’t. They chose to tweak the development environment for OSX to include the iPhone, and by necessity, also putting OSX on the iPhone. The result of this is that if you want to develop for the iPhone, you have to get a Mac (strike 1), learn OSX (strike 2), learn Objective-C (strike 3), learn Cocoa (strike 4), and by then you’re so deeply immersed in the Mac environment that you won’t find your way out again. Since you can run your Windows stuff, including Visual Studio, just fine under Parallels or Fusion, you don’t need that Dell or HP machine for anything anymore, and you’re not sorry to see them go. In other words, you’ve got a developer that clearly isn’t going to like going back to .NET development again. I mean, once you’ve used these two environments (Xcode/Cocoa/Objective-C vs .NET/Visual Studio) it’s practically impossible to enjoy .NET anymore. It’s so far behind and so very clunky in comparison it’s almost a joke.

So, every developer you task with iPhone development is almost certainly lost from the .NET camp forever. This I can’t prove, but I’m convinced of it. But now is the question: who are these developers? Do they already develop for the Mac or are they from the “other” side? Again, by the seat of my pants, I’m convinced that a very large and increasing proportion come from large enterprise .NET development organisations that need to add a client for their large systems on the iPhone. See where this is going?

It’s only just begun.

Update: I suddenly realized that I fused two unrelated events together in my mind. Steve Ballmer did the monkey dance and yelled “Developers, developers…!” at two different, equally traumatizing, occasions. I’m not sure that’s any better, though. It’s all very disturbing.

The Microsoft .NET Framework 3.5 Service Pack 1 update, pushed through the Windows Update service to all recent editions of Windows in February 2009, installs the Microsoft .NET Framework Assistant firefox extension without asking your permission.
This update adds to Firefox one of the most dangerous vulnerabilities present in all versions of Internet Explorer: the ability for websites to easily and quietly install software on your PC. Since this design flaw is one of the reasons you may’ve originally choosen to abandon IE in favor of a safer browser like Firefox, you may wish to remove this extension with all due haste.

Unfortunately, Microsoft in their infinite wisdom has taken steps to make the removal of this extension particularly difficult – open the Add-ons window in Firefox, and you’ll notice the Uninstall button next to their extension is grayed out! Their reasoning, according to Microsoft blogger Brad Abrams, is that the extension needed “support at the machine level in order to enable the feature for all users on the machine,” which, of course, is precisely the reason this add-on is bad news for all Firefox users.

And then follows a convoluted procedure to hack the crap out of the registry. Go there, read it, do it, if you run Windows, this service pack, and Firefox.

Tech Republic put it like this:

In a surprise move this year, Microsoft has decided to quietly install what amounts to a massive security vulnerability in Firefox without informing the user. Find out what Microsoft has to say about it, and how you can undo the damage.

Read the entire Tech Republic article.

PS: this isn’t exactly news (the annoyances.org article is dated February 27, 2009), but I only just noticed through a posting by Rob S on a private list.

When using the asterisk wildcard character in a searchPattern, such as “*.txt”, the matching behavior when the extension is exactly three characters long is different than when the extension is more or less than three characters long. A searchPattern with a file extension of exactly three characters returns files having an extension of three or more characters, where the first three characters match the file extension specified in the searchPattern. A searchPattern with a file extension of one, two, or more than three characters returns only files having extensions of exactly that length that match the file extension specified in the searchPattern. When using the question mark wildcard character, this method returns only files that match the specified file extension. For example, given two files, “file1.txt” and “file1.txtother”, in a directory, a search pattern of “file?.txt” returns just the first file, while a search pattern of “file*.txt” returns both files.

and:

Because this method checks against file names with both the 8.3 file name format and the long file name format, a search pattern similar to “*1*.txt” may return unexpected file names. For example, using a search pattern of “*1*.txt” returns “longfilename.txt” because the equivalent 8.3 file format is “LONGFI~1.TXT”.

The conclusion must be that this function is worse than useless and bound to cause excruciating bugs in your apps. Better use the GetFiles() method without any search pattern and then filter using a regex.

.NET development is single platform. Admittedly the largest platform, but a platform that is increasingly having to share the market with other platforms. And already, according to some, there’s more sales potential for small developers in the OSX market than in the Windows market, due to a number of factors like customers that are more willing to buy and to pay for software, less competition in each market segment, etc.

.NET development is also entirely dependent on Microsoft’s development tools and those are increasingly expensive. For reasonable development, you need an IDE, a good compiler, version control, bug handler, coverage analysis, profiling, and a few more. We used to have most of that in the regular Visual Studio, but recently MS has removed all the goodies and plugged them into the Team system only, which carries an obscene pricetag (in Sweden around USD 13,000 + VAT for the first year…). This means that a regular one-man development shop can barely afford the crippled Visual Studio Professional at USD 1,500 for the first year. Sadly, there aren’t even any decent and affordable third party products to complement the VS Pro so it becomes a “real” development suite. And with every version of Visual Studio this only gets worse. More and more features are added to the Team suite and removed from the Pro. This is not the way to breed a happy following.

Meanwhile, OSX comes with XCode, which is almost as good as Visual Studio Pro, and is free. Objective-C is also a much more modern language with more depth than any .NET language, even though it is actually older. But, sadly, it’s not cross platform either and I don’t see how you can get the Windows fanboys of the Scandiavian healthcare scene to even consider another platform. Same probably goes for most other industries.

I’m no fan of Java, but on the other hand I’ve never worked much with it so that opinion doesn’t count. Eclipse, the IDE often used for Java development, is cross platform, very capable, and open for other languages such as Python, Flex, and many more. Yes, I know, in theory so is Visual Studio, but how many real languages do you have there? You’ve got one: Basic, masquerading as C#, J#, and, um, Basic.

Using Eclipse on any platform, you’ve got a real good chance of covering the line of tools you need, profilers, coverage, version control, without much pain and without breaking the bank. And you can write crossplatform integrated larger systems.

So, I guess it’s time to bite the bullet. I really like XCode and OSX, I really know C# and .NET, but I really only believe in Java, Flex, Python, Perl, C++ under Eclipse for enterprise development in vertical markets. And in XCode under OSX for regular shrinkwrapped desktop apps.

Not even Silverlight is very attractive and that is largely due to the marketing and pricing of the tools for it. A small developer organisation can’t afford it. Flex and AIR looks like serious contenders, though.

x2c stands for “XML to Code”, and it’s an interpreter for a little language I made with built-in commands to handle XML documents and write to plain text output files.

It started life as a tool to create VB and C# source code for data access layer classes, based on XML descriptions of an Oracle database. Another possibility is generating language tables from Excel spreadsheets, and I’ll tell you how:

Imagine an Excel spreadsheet with one sentence per row. In each column, the same sentence is written in another language, like Swedish, English, French, etc. Save the spreadsheet as an XML document. Now you can write a pretty short x2c script that reads these languages, column by column, and then produces a C++ header file with the right strings declared as constants. Great for products you want to recompile for a number of human languages.

Especially for this last use, I recently adapted the text output file command in x2c to allow output to ASCII, unicode (default), or any codepage you have installed on the Windows system you’re running this thing on. In the above script example, you see codepage 1251 used for Russian. In this case this was necessary since the C++ compiler used (Borland) couldn’t use unicode header files. This script runs under US or Swedish XP and Vista, as long as codepage 1251 is also installed on the system, and then produces the right MBCS file for Borland C++, resulting in binaries that will look real to russians running russian versions of Windows. Note that above is the complete script that is needed to convert the excel spreadsheet to four different C++ header files and it can easily be run from a build script.

The source is C++ in a VS 2008 solution. Have a go at it.

Any solution also needs to pander to intellisense, making it easy to use and kinda idiot safe (I’m talking about myself a couple of hours after defining any constant, which usually leads to me behaving like the idiot user I had a hard time envisioning just hours earlier).

I think I found a good system for doing this, and as an example, I’ll invent a function that takes a string parameter, but it has to be just the right kind of string. To do that, I first declare the constant strings in a separate module this way:

Then I write my function, the fictional “Rechandler” that takes a parameter of the ConstRecTypeValue kind. And then I write a function that calls it. Now, while writing the caller, I want intellisense to do its thing, and it does:

As you can see, it obediently pops up a tooltip to tell me only a ConstRecTypeValue is accepted here. As soon as I start to type that, it recognizes the ConstRecType static class name and it intellisensively lets me choose which constant member I want:

…which I complete the usual way:

The callee (Rechandler) then easily recovers the string that is hiding inside the passed value (in this case “DELETED”) and continues its merry ways.

Naturally, you can use chars, doubles or entire collections of values instead of the string value in this example and still achieve the same effect.

You can also take it one step further along the path to universality, by using a generic base class for the value type:

If you have this guy in reach in your project somewhere, you can now simplify the definition of the value class like so:

…while everything else stays just the same.

I love it.

But using reflection in C#, you can easily make it so the system retrieves all your constants and their values into a dictionary at runtime and saves them back, using nothing but the declaration of the string constants.

This is an example of a declaration of the names of the values we want to save and restore:

And this is code that then gets the values of those constants and sticks them into a dictionary in runtime. The rest of the code is trivial and not worth reproducing here