Handmade Hero

Just watchted this rant and as I listen, certain obvious facts became clear to me that Casey is completely missing:

1) It's pretty obvious why Microsoft cares little about the load-time of VS. Their main clientell tend to just have the thing open all the time. They don't open it many times per-hour, just for debugging. It's a heavy-duty IDE, not a text editor.
So that workflow is just not on their radar as something to be concearned about - at all.

2) Software has a maturation cycle, it peaks at some point and then gradually deteriorates. It's a bell curve.
That is ESPECIALLY true for any long-lived commercial software. Just take any software that you can think of that had existed for over a decade, and you'll see that dynamic at play.
The reasons for that have nothing at all to do with developer competence.

The main reasons are economic, and psycho-social.

The economic reason is akin to "planned-obselecense" in consumer electronics:
They can't just develop something to perfection, and then just continue selling the exact same thing every year, because consumers will then have no reason to ever upgrade. With consumer electronics they can time some components to fail after some period of time. But software doesn't deteriorate if you just keep using it without change.
So unless you do something nepherious, the only way to get people to upgrade is to add more features all the time - even when they are completely unncessary, or are relevant to very few people.
That's why commercial software gets bloated over time.

To some extent, open source software suffers much less from this, as it can affort to stop evolving once it's reached some zenith point, doing what it needs to do well - and without any negative economic consequences.

The psycho-social reason has to do with scale:
The bigger a project gets and the more people are involved, the more likely it gets that the product will get worst over time. That's true about any kind of large scale project, and is not even specific to software.
The project accrudes more varying oppinions over time and so more likelyhood for conflicting ones, pushing the project internally into different directions at the same time. The larger any group of people get, the more politics there will be. It's inevitable.

Take the case of C++ as an example: "Design by Comitty".
Now contrast that with what J.B is doing:
Even within a single person's mind, there will inevitably be contradictions and competing interests, and more and more of those as a project grows and expands. Because there more variables there are, the higher is the likelyhood for conflict. That's just basic combinatorial logic.
Contradictions are far more likely to be resolved within the privacy of a single person's mind then within a group of people, and the larger the group the lower the likelyhood.
So the more people are involved in a project, the higher the likelyhood for more internal contradictions in it.
That's a major contributer to incidental complexity.

I can go on about the accrusion of technical debt, the explosion of "special-cases" as a project accrudes more features, the necessity for backwards-compatibility, etc, etc.


But really, these are some clear reasons, none of which have anything to do with developer competence.

One more thing:

The comparison with George's RemedyBG in the rant is completely backwards(!)

He doesn't have to deal with huge amounts of internal politics.
He doesn't have to deal with multi-decade accrusion of: technical debt, bloat, cruft, incidental complexity, etc.
He doesn't have to deal with a huge huge software that does a million other things that have nothing at all to do with what he wants it to do, and that probably have systems in place that are a straight jacket for his ability to produce quality software.

The likelyhood of any single highly-competent person to produce a higher quality product, thatn ANY group of even very higly competent developers - IN SUCH A CIRCUMSTANCE - is actually VERY high.
If anything, it should be a huge surprise if he DIDN'T manage to produce something of higher quality.

The circumstance that the VS developers at Microsoft find themselves in, is most probably none of their own making.
They probably just inherited this huge pile of mess, and are probably doing the best they can in an impossible situation.

And for the record, I'm using CLion, and even though it's sometimes slow, and buggy, I still like it.

I don't agree what you're talking about is orthogonal to developer competence.

For your first point: have you ever needed to work on several projects at the same time? I do right now, and in these circumstances, you often need to boot-up Visual Studio several times a day. It's not as retarded as something like Unity which allows you to only have 1 instance open at all times, making working on several projects at the same time even worse, but in the case of VS, you have to deal with its high memory usage, sometimes high CPU usage in an instance you're not using, or with it suddenly deciding to stop allowing you to save your progress or straight up crashing, leading to you needing to kill it and launch it again and again.
And the startup-time is not the only thing they don't care much about: try comparing the results of the feature "Find all references" on C# code, and then on C/C++ code, and you'll conclude they don't care anymore about C/C++ development.

Your second point sounds like what Jonathan Blow talks about when he complains about the lack of "adult supervision" in modern software, which IS a developer competence (or should be).

You're talking about the deterioration of software as if it's a natural process, like an old car whose pieces get more and more rusty with time. But even in this case, the "adult" thing to do (and costly, of course) would be bringing your car to a competent mechanic to get it to last ten more years.

I have actually worked in a team that redone their software almost completely several times, even though they still generated revenue, to have a good base for the future. For example, they had a version 2 with a rushed undo/redo feature that worked only half the time, so for version 3, they redid part of their architecture so undo/redo could work without trouble for everything. On that same project, I had to add a whole new feature that required its own UI, some threaded baking of geometry and even a bunch of network accesses, and adding all that stuff didn't impact startup times or performance for users who didn't use that feature, because why would it? (And still, when I look back at my work on that, I consider it sloppy...)

I don't understand how you came to think that open-source software is less a victim of the obsolescence you're describing? This is certainly true for small and focused libraries (who ever had to update the zlib in his project?), but just look at the web stack, which is a pile of open-source technologies, in which trends change every two years for no real reason.

The only "good" reason I've seen for software to get worse is when it has a monopoly and will still generate revenue, even when all its users complain constantly about it because they still need its ecosystem.
I'll take Adobe Illustrator as an anecdote: my girlfriend currently works at a job which requires to have some Illustrator plug-in to interface it with some industrial printer. Since Adobe is pretty much the only relevant player, the printer's manufacturer only created a plug-in for Illustrator, so even if she wanted to switch to Affinity Designer or something else, she can't because of her job requirements.

"Adult supervision" is also not letting junior developers get in charge of a new product or of the architecture of the current product. I worked in a game company where most of their game engine was made by fresh out of school engineers (when I was still at school, so I'm not judging here), and when they look back at it now, they see it's unjustifiedly OOP and at some places hard to debug, and also does things that no "professional" game engine does, like relying on un-baked assets even in release builds.

Also, "adult supervision" is about something we advocate for all the time in Handmade Network: measure, measure, measure.
If you have several conflicting ideas, do them all and measure which one runs the fastest for your current needs. Don't judge them by their philosophical prettiness or by how they will give you more flexibility in a hypothetical future. Get an automated test system that sends an e-mail to every developer if the startup time of your product gets seconds too long.

Interesting points. I think Arnon is right that the issues with VS are more likely due to social problems than developer incompetence. It's hard to convince managers and executives, even at the smallish startup where I work, to let me spend a sprint on load times. There are always 100 more important-sounding tasks (and honestly, tasks that probably do more directly translate into revenue, or customer happiness, or whatever). Performance is always death by a thousand cuts anyway, and it's hard to quantify up front what the issues are, so it's hard to plan out, etc., etc....

All that to say: I am sympathetic to the Visual Studio team and trust that they could absolutely destroy those long load times if they were tasked with doing so.

BUT. I think the comparison to RemedyBG is still a fair one from a pure consumer perspective. The difference in absolute performance is staggering. I think it's a strong argument in favor of smaller tools and high-quality work over tools that try to be everything to everyone.

I know that from a purely technical perspective, Visual Studios long load times are an embarassment. But I don't think this is fundamentally a technical problem, and while I'm all on board with a healthy rant about how slow the software is, I'd like to make sure the anger is directed at the right people.

Also, props as always to George for demonstrating how good things could be.

Guntha
have you ever needed to work on several projects at the same time?

Yes. That's very common for me. I just have multiple instances of whatever IDE I'm using (typically of JetBrains).
Memory is plentifull and cheap. I don't care if it takes 2GB.

Guntha
try comparing the results of the feature "Find all references" on C# code, and then on C/C++ code

I never use VS in int's vanilla form. I always have some plug-in for C++. Visual Assist used to be the standard.
Recently I tried "Resharper for C/C++" (from JetBrains), and I liked it a lot. So I don't know what the vanilla perf. is for stuff like that, because I don't use that. I use whatever comes with the plugin I'm using.

Guntha
You're talking about the deterioration of software as if it's a natural process, like an old car whose pieces get more and more rusty with time.

Not really... As I've just said:

Arnon
But software doesn't deteriorate if you just keep using it without change.

It's a natural process for ANY large system to devolve when a large group of changing people are in charge of it.
Has nothing at all to do with software. Just look at the legal system, or goverments. That's what I was saying.

Guntha
I don't understand how you came to think that open-source software is less a victim of the obsolescence you're describing?

It not always is. I've said:

Arnon
To some extent, open source software suffers much less from this.

I was talking about examples like the one you gave: A dedicated tool that does one thing and does it well. In the linux land there are plenty command-line tools that are examples of that. But again, this is about economic incentives. Free software doesn't have many of the constraints and incentives that commercial software has - that was my point.

Guntha
"Adult supervision" is also not letting junior developers get in charge of a new product or of the architecture of the current product.

I totally agree and resonate with J.B and C.M's about their frustration with the lack of what they call "Adult Supervision".
It's kind of a very high level term though, so it can mean all kinds of things to different people.
Seniority level is not the exclusive measure of quality of software developers - it's very possible for someone to be very senior and at the same time a horrible software developer and/or leader of software projects.
I have experienced first-hand the kind of bad cases of lack of adult supervision in my recent job.
(I'll elaborate more in a bit on what "I" consider to be the role of leadership, and when it'd not doing it's job properly)

Guntha
If you have several conflicting ideas, do them all and measure which one runs the fastest for your current needs.

I was more reffering to overall architectural design descisions, general directions, etc. It's virtually impossible to measure the impact of those, because they span across the entire code-base, and take years to evolve. It's easy to measure performance of some small constrained portion of the code, but that's not what I'm talking about.

Initial loading is not a small and constraint kind of process - everything in the entire software stack revolves around how things are being done there. And a lot of run-time performance can typically predicate on paying a higher cost at load-time, once, for the sake of winning better performance during the entire lifetime of the process - residually.
The more systems a software has in it that wants to have that benefit, the more cost will be accuded at load time.

One of the main jobs of a good leader, is to identify the edge-cases - the people on either end of the spectrum of contibution vs. damage to the product/company. Because the distribution there is non-linear.

It's very "non politically correct" to say it, but nevertheless it's VERY true in most case, that:
In a large organisation, there will tend to be some very small percentage of people who contribute the vast majority of the benefits to the product/company, on one end of the spectrum, and on the other, it's typical to have a different small precentage of the people who cause most of the damage to the product/company.

So you got a non-linear curve on each end of the spectrum.

Your role, as a leader, is to identify the few people who cause most of the damage, and somehow neutralize their ability to do so. At the same time, identify the few people who contribute most of the value, and empower their ability to do so.

Another VERY common issue - in ANY field, not just software - is this:
The's a hierarchy of competence, and then there's a hierarchy of authority.
Sometimes they go together, often they don't - and that's not always bad.

If someone is a brilliant engineer, they are most likely much more interesting in "things" than "people".
That's kind of why they are good at engineering.
But it often also makes them terrible managers.

Say you want to compensate them somehow, or empower them - because you wan to keep them around, and you want to extract even more value out of them. What's your options?
A one-time bonus?
A salary raise?
A public appreciation?

A promotion?

If you promote them, do you make them into a manager?
If you do, is that the best descision you could have made?
What if they don't want to manage people?
What if they're not good at it?

You'll be damaging the company, probably damaging them, and the project they'll be in charge of.
You'll be loosing a lot of value that they could have still been able to produce if they remained engineering.

That's a general problem, not particular to software

The problem should be that salaries should be de-coupled from the authority heirarchy.
But that almost never happens.

This conversation seems interesting and thoughtful, but I just want to point out that the only points Casey was actually making were that Visual Studio is dead and that an incident report won't fix that. RemedyBG has already overtaken it in all the metrics he feels matter, and there's no sign of that reversing any time soon.

Fair enough, I wasn't trying to "convert him back" (or anyone else for that matter).
People should use whatever tool suits them best.

I just felt that the reasons he gave for why VS is slow, were illegitimate, and that he was being unfair towards Microsoft developers of VS - especially when comparing it to RemedyBG.

And again, I'm not a VS fan by any stretch, I just know what circumstances can arise with large software projects that survive for decades at large companies. I guess that causes me to have a different set of expectations.

I don't expect VS to ever launch at the speed that RemedyBG.
But I know that this has waaaaay more to do with the fact that RemedyBG is a green field project done by one person with a very narrow focus, than anything whatsoever to do with either the competence levels of developers at Microsoft, or them carying about it or not. It's actually very possible that some of them DO in fact care a lot (whether they express that appropriately in public or not), AND they are extremely competent, AND at the same time there's absolutely nothing anyone can ever do about it at this point.

I'm using CLion, which is built on the core of IntelliJ, which is a very large Java stack, primarilly targeting Java development, which is also a very large project that has evolved in a very large company for many years.

It probably takes even longer to load than VS - and you know what? I don't mind it. You know why? Because I don't do that many times an hour, and I don't have any expectations for it to be fast.

I think it's important to manage your expectations appropriately - otherwise you're just settings yourself up for frustration.

With all that said, I'm totally in aggreament with J.B and C.M about the catastrophe that the software field at large is in this day and age, and that this probably has a lot to do with software culture around performance, with respect to certain commonly heralded practices, methodologies and programming languages. It's a disaster. Yes. It should be fixed. Maybe it can be. I don't know.
We'll see if Jai makes a dent in that. but even in the optimistic case that it does, that will not immediatly erase many decades of old crufty bloated slow code. Or a decade of garbage javascript around the interweb.
And even after that, I expect Jai to be horribly missused, and for very slow software to continue to be produced - even with it. Especially for large projects - even if they are ported over.

And in a decade's time after Jai gains hold (assuming that it does), projects made in it will again suffer the same fate as VS, and again - for many reasons that have almost nothing to do with competence, and everything to do with all the other factors that I've detailed in this thread.

So, again: Expectation management is the key to sanity there.
Just my 2 cents on that.

It's pretty obvious why Microsoft cares little about the load-time of VS. Their main clientell tend to just have the thing open all the time. They don't open it many times per-hour, just for debugging. It's a heavy-duty IDE, not a text editor. So that workflow is just not on their radar as something to be concearned about - at all.

I don't buy this excuse - at all. Beyond being plainly untrue as Guntha mentioned, consider the counterexamples. For example, Blender is WAY more complex than VS and has WAY more functionality, and is also intended to be kept open for long session, but it starts up in 2 seconds. It's perfectly possible and reasonable for a piece of software to be very complex without being "heavyweight". Blender has also been around for 26 years (a year longer then VS!), is accumulating new features constantly, is very much in the multifunctional "everything to everyone" camp, and yet still clearly improves with every version.

If you want an economic or psycho-social reason to point to, I think Guntha is right on the money. First, it doesn't monopolize the 3D software space - on the contrary, it's a competitor to the bloated software that used to monopolize it, and still does monopolize some parts. Second, the core team is very selective about who they bring on board, and the overall course of development is being steered by highly experienced people who have a relatively clear goal and know what they're doing - in other words, "adult supervision".

Furthermore, there's a chicken-and-egg problem regarding startup time (and performance in general) that you're not accounting for. Is the software slow to start because people keep it open for long sessions, or do people keep it open for long sessions because it's slow to start? I'd bet it's MUCH more of the latter than you think. The workflows people develop with software are shaped by how that software behaves at least as much as by the work they're trying to do - especially if they don't have prior experience with software that behaves differently.

In a large organisation, there will tend to be some very small percentage of people who contribute the vast majority of the benefits to the product/company, on one end of the spectrum, and on the other, it's typical to have a different small precentage of the people who cause most of the damage to the product/company. Your role, as a leader, is to identify the few people who cause most of the damage, and somehow neutralize their ability to do so. At the same time, identify the few people who contribute most of the value, and empower their ability to do so.

Sounds like you're saying it has a lot to do with developer competency...

There's a difference between individual competence and cultural competence.

You can find individually competent people at any company, but they can still be part of an incompetent culture. These are the people who can help you get the VS startup time down to "under 10 seconds", but won't consider that it should actually be near-instantaneous, because that sort of thinking is not part of that culture's mindset. If you could transport those people into a competent culture, they would be very useful there as well (and probably a lot more productive).

If you have a competent individual that is part of a competent culture, but is working at a company that is largely made up of programmers who are part of an incompetent culture (regardless of the programmers' individual competence), they would be pretty miserable. It would be hard to do good work, and there would be a lot of pressure to join the incompetent culture.

A competent culture pushes people towards individual competence. Beginners join and are welcomed, and if they adopt the culture's mindset they will eventually become individually competent.

By contrast, an incompetent culture doesn't make any demands and doesn't have any expectations. People might become competent, but the culture doesn't know how to evaluate that. An incompetent culture pushes people towards busywork instead.

This creates some nasty feedback loops. An incompetent culture is more likely to attract beginners, because it imposes no pressure and no expectations. As a result, it's easier for an incompetent culture to grow and to nearly take over the industry. At that point the incompetent culture sets the standard for software quality, which makes a competent culture seem unreasonable, which drives more people away from it. Now we're at a point where even if you wanted to start a company with a competent culture, practically no one in your hiring pool is part of that culture.

The reason you sometimes see projects made by individuals or very small teams that are way better than their large-company competitors is not necessarily that those individuals are highly-competent (though they might be), it's just that they have a competent culture's mindset, and that's easier to achieve for an entire project when you only have a few programmers. And so, even if these people were not highly-competent at the start of the project, they will become individually competent at the end because a competent culture pushes people to be individually competent.

notnullnotvoid
Blender is WAY more complex than VS and has WAY more functionality, and is also intended to be kept open for long session, but it starts up in 2 seconds. It's perfectly possible and reasonable for a piece of software to be very complex without being "heavyweight". Blender has also been around for 26 years (a year longer then VS!), is accumulating new features constantly, is very much in the multifunctional "everything to everyone" camp, and yet still clearly improves with every version.

Blender is somewhat of an anomaly in this space: Ton Roosendaal has stated many times that the loading time of blender is "a major feature". That has been a mission-statement from the get-go, and all througout Blender's lifetime. So it doesn't come as any surprise to anyone that knows that, that Blender loads fast. Every descision made in the development of Blender, has this feature accounted for, so there's been continuse significant effort to keep the loading time low.

This has a lot to do with how Blender has a single BDFL (like python) all thoughout it's existence - something you NEVER get in ANY commercial software. Someone who can veto-out certain descisions that would negatively impact Blender's loading time, and that sticks around for decades.

Name me a single commercial software product that has that....
If that's how you define "adult supervision", then it should be no surprise why commercial software always ends up slow - because no commercial company can ever have that.

Add to that all the things I said about the differences of commercial vs. free software: Especially how there is no commercial incentive to spit out a new version each year to satisfy share-holders, and with it a pressure to add bloat in the form of new features that almost nobody needs. Blender can very well allow itself to even stagnate for very long time durations, something that NO commecial software can ever afford.

These are the main reasons why Blender loads fast - not just the competence of it's developers.
Economics and culture - which are somewhat tied together.

notnullnotvoid
there's a chicken-and-egg problem regarding startup time (and performance in general) that you're not accounting for. Is the software slow to start because people keep it open for long sessions, or do people keep it open for long sessions because it's slow to start?

Sure, there's that aspect of it.
But there's also:
1. Users keeping instances of their project up for a long time because they are working on them for a long time...
2. Commercial software becoming slow to load for all the reasons I pointed out, having nothing to do with workflow.