My Thoughts on LLMs

I've been trying to write a positive blog post about using LLMs as a senior developer. I've made a couple of starts: one looking at how an LLM helped me solve a problem when documentation came up short, and one about how you should use the LLM to generate code that you're familiar with before using it in an unfamiliar space.

In both cases, I got bogged down in “this is just me telling a fishing story,” and nobody wants to read those. I also found that the second had a lot of my thoughts on LLMs in general. So let's just put those into their own post. It may never get beyond my computer, but I still need to write it.

TL;DR: I have zero desire to spend the remaining few years of my career as amanuensis for an LLM that's spitting out code. But I think they're incredibly useful as an investigative tool.

Let's deal with the elephant in the room first

Ethics, and more importantly, whether you can use an LLM ethically.

And I'm going to start by making some people mad, and point out that ethical questions always have ambiguous answers. If they didn't, then the trolley problem wouldn't be a problem. Every human has their own concept of what is right, what is wrong, and what is wrong but justified. My ethics are not your ethics. And this post reflects my ethics.

And as you read this, you may say “that's just whataboutism!” And you would be 100% correct. I don't believe that whataboutism serves as a defense — two wrongs don't make a right — but it's an invaluable tool to understand where I (or you!) draw an ethical line. Is something different in type, or just scale? And if it's only different in scale, what makes one scale ethical and another not?

Lastly, I'm going to focus on the issues that I think directly impact using an LLM for software development. There are a lot of ethical issues outside of that narrow use case, but I'm going to ignore them. Much the same way that I ignore that ammonium nitrate can be used for bombs as well as fertilizer.

Resource Consumption

One of the ethical arguments that I see constantly is that LLMs consume enormous amounts of electricity and water; that you're &lduo;burning the planet” to use them. For me, this is the reddest of herrings, and I'm going to bury it quickly. I'm writing this in an air-conditioned room, on a computer that runs 24/7, surrounded by products of the modern industrial age. Yes, I try to live a lifestyle that minimizes my impact, but there's only one way to reduce my resource consumption to zero, and I'm not willing to take that step.

Whether resource usage is “good” or ”bad” is a matter of individual perspective. I think that the resources that are consumed to keep my house comfortable in the face of Philadelphia's summer heat and winter cold are justified. You may not, but unless you're living off-grid (and not polluting the air with a woodstove) I don't see that you can claim the moral high ground.

And yes, I realize that the GPUs used to run LLMs consume more power than CPUs. And that they require more copper to provide that power, and more cooling to take away the heat of computation. But that's a matter of scale, not kind.

As such, I believe it's ethically sufficient to limit my usage of LLMs to where they provide value that is not readily available from other means.

Copyright

I think copyright — creators' rights in general — is the bigger concern. I believe that creators should have the right of control over the uses of their creations. And that others may not take unilateral action to abridge that right. That's not quite the same as copyright: in some cases it's more stringent, in others less.

How then, can I reconcile the LLM builders' indiscriminate scraping of the world's content in order to build their models?

The answer is that I carve out a giant exception when the creator puts their work out for others to freely consume. As, for example, I did with this blog post. There's a notice at the bottom that says it's copyright by me, with all rights reserved, but it's posted on my website where it's available to anybody. Be they human, an LLM bot, or — critically — a search engine bot.

And that last one is the reason for my carve-out. The web as we know it has been around since the mid-90s. In almost all of that time, there have been search engines scraping publicly accessible pages (and some that aren't meant to be public), indexing pages by the keywords found within. And storing the literal page text, or at least excerpts of it, to provide context in search results.

And in that time, those of us who style ourselves creators have accepted these bots, and the search indexes they produced. At least mostly-accepted: there was an outcry against Google scanning and indexing the world's books back in the mid-2000s, but it didn't seem to go anywhere. As of this writing there's still a Google Books site, where you can search “the world's most comprehensive index of full-text books”.

Clearly, search indexes provide utility for the consumer. I remember the world of software development before the web, when every developer would have a bookshelf full of technical books, and manually search them for answers. And to geez a little bit: in general, pre-web documentation was much better than anything that came afterward, when the web spider would extract keywords and you could bounce all over the documentation (and independent postings on the web) looking for answers.

But search engines also provided utility for creators: they opened our work to a wider audience. A relative nobody (me) could find himself referenced by a widely read technews journalist, simply because his article on an estoric topic was at the top of the search rankings. Or contacted by in-house recruiters for well-respected companies, again because their search queries turned up my work.

For the past 30 years, I haven't given much thought to the relationship between search engines and creators. But the backlash againt LLMs have made me think about it more. How are LLMs different from the search engines that I already accept?

Where I'm landing is that LLMs have broken the implied relationship between creators and search. The “AI summary” at the top of modern search results exists to keep you from clicking through to the actual pages. So for me, the ethical answer is to click the link, to use the tool as a gateway rather than an endpoint.

This doesn't quite always suffice, as I'll consider later when I look at LLM benefits.

No, Really, Copyright

OK, but what about actual copyright violations? Let's start by pointing out that copyright is a legal term, and while law is (hopefully) based on ethics, it does not define what is ethical. Breaking a law is not, per se, an ethical violation.

Moreover, I am not a lawyer, but I've been watching what lawyers do — and more importantly, don't do — about LLMs and copyright. And it seems that LLMs are being considered “fair use” of the source material.

This was the case in the class action suit against Anthropic: building the model was fair use, but building it from sources downloaded from pirate sites was a copyright violation and entitled authors to compensation. And on the other side, the Free Software Foundation has not chosen to bring suit for violation of the GPL. It seems to me that they would do that if they felt that the could show that an LLM was a derived work of GNU-licensed software.

As a non-lawyer, this seems reasonable to me. The original content exists nowhere within the model. It's just statistical assocations between words, other words, and whatever tags the creators chose to apply during training. So you can ask the model for a Python module that logs events to CloudWatch, and it will put together the words that are strongly associated with those tags as well as each other. And you can ask it for a short story about baby shoes in the style of Hemingway. But in neither case is it simply regurgitating an existing work.

Where does fair use end? That line has certainly moved since George Harrison was successfully sued for copyright violation because My Sweet Lord had the same melody as He's So Fine. That had always seemed a stretch to me: musicians have long drawn inspiration from each others works, consciously or unconsciously.

As a point of law, this question will ultimately be settled in court. For me, the ethical question of whether creators control their work is more important than the legal question of whether the models violate copyright.

For what it's worth, I'm also OK with archive.org making old magazines and books available for public consumption. But much less comfortable with their “National Emergency Library”. As I said, ethical questions are all about where you draw the line.

The problems I see with LLM usage

For this section I'm going to focus on the use of LLMs in the field of software development. And while I see lots of issues, I'm going to limit myself to the three issues that I think are most important.

Reliance on an Unreliable Resource

By this I don't mean that the output of an LLM is unreliable, but that the LLM itself may not be available when you want to use it. For example, on September 3rd, 2026, all four of the major models experienced service interruptions. If your application — or your developers — can't work without connecting to a model, you're putting your business at risk by relying on them.

Subtle Bugs

I'm really quite impressed by the ability of an LLM to generate decent code. When I was working on the “know your LLM” post, I used prompts that were intentially vague, but from a problem space that has plenty of examples available on the Internet. And the results were quite impressive: the code was well-structured, and it worked.

But it had a race condition. Or to be more correct, it had a race condition three out of the four times that I used the same prompt. The fourth implementation didn't have the race condition but used a very inefficient way to solve the problem.

I reviewed this code very carefully, because it was going to be a key part of a blog post. If it was a normal pull-request, I wouldn't have reviewed it quite so closely: my normal approach to PRs is to look at the logic and flow from a high level, and spot check things that I know might be an issue. And faced with a flood of incoming PRs, I might be even less cautious than normal (I haven't been in this situation, so don't know).

Some people will respond that tests will solve this problem. But I have a lot of experience writing tests, and know that good tests — the ones that probe the edge cases — are extremely difficult to write. They take far more effort than the code being tested. If you're using an LLM to generate your mainline code, are you likely to expend the extra effort to manually write tests?

Destroying the Internet as a Source of Knowledge

I recently wanted to sanitize the fresh water tank on my travel trailer. It's something I do two, maybe three times a year, and I can never remember how much bleach to use (for my own future reference: it's 1/2 cup for the whole tank). So I did a web search. And at the top of the search results was a site that told me to use 1/2 cup per gallon of water. And it helpfully extrapolated that, using the wrong tank size, to 12 cups of bleach for the whole tank. That's enough bleach to cause some pretty serious damage to the plumbing system.

Was that site generated by LLM? I believe so, even though it wasn't quite as smarmy as the usual slop. I suppose it could have been created by a person who had no clue what they were writing about, and just made stuff up. But there are many other examples of sites that are clearly LLM-generated, with text that's designed to keep the reader occupied for as long as possible without actually giving them the information they came for.

And, unfortunately, search engines eat that stuff up and push it to the top of their rankings. It checks all of the boxes of what Google tells us makes a “good” web page: in-depth (or at least long), contains lots of detail (or at least repeats its main point multiple times in different ways), and will adjust its formatting to be readable on mobile devices.

This may have a bigger impact on the non-programming world, as slop gets mixed with intentionally wrong information, burying the truth. There are, after all, only so many ways that you can write code that works. But if you are reviewing a PR and see something that doesn't look right, how will you know the truth in a world of LLM-generated references?

Where I think LLMs are valuable

After the last section, you may think that I'm dead set against LLMs. But in fact, I think that — at least in the software development realm — they offer some real benefits. Just not as the primary tool for writing code.

Prototyping

This topic comes up in my conversations a lot: LLMs let non-technical people prototype their ideas. And if that prototype doesn't quite do what they want, they can re-prompt until it does. Or until they give up and turn the whole thing over to a professional developer.

This was one of the promised benefits of “Agile”: non-technical people could see rapid feedback. If they didn't like what they had asked for, it could be changed. Sometimes, this would get into a frustrating “I'll know it when I see it but that's not it” loop, but usually a few iterations were sufficient. And that meant that they weren't disappointed when the project was officially released.

Of course, the risk with prototypes is that many non-technical users think that the work is done when the prototype looks right. When, of course, the real work is just beginning. I'm hopeful, however, that the need to re-prompt to achieve their goals might give those users the idea that maybe, just maybe, it isn't quite so simple to develop software.

Discovery

On the opposite end of the development life-cycle you have legacy codebases. They've grown organically over time, and have been modified by many people of greater or lesser skill. As a result, they are usually difficult to understand, and developers avoid difficult-to-understand code because it's easy to break.

An LLM provides the ability to analyze that codebase, identify major areas of functionality, and drill down into exactly how it works. I have friends that have used it to analyze large codebases, and swear by the technique as a first step to modifying those codebases. I've used it on smaller codebases, that I'm familiar with, and have been impressed with what it can do. But your prompts are critical!

One example: around 14 years ago, I wrote a tool to extract endpoints from a Spring web app. I stopped working with Spring, but it (Spring) continued to evolve, and now there are more ways to specify endpoints, which the tool doesn't support. Simply prompting Claude to explain how the tool processes Spring apps gives a good overview of what it does now, but doesn't say anything about missing support for those new features. You have to know that a feature is missing, then give Claude a very specific prompt, such as “does this tool support @RestController”, after which it will give a good explanation of why it doesn't, with pointers to what needs to change (and it will offer to make the changes for you, but see above for why I wouldn't do that).

Bootstrapping

There are a lot of general skills in software development, from how to structure a program to whether you should use a list or dictionary in a particular algorithm. And there are a lot of specific skills, that may have language- or framework-specific idioms. For example, how to access a database in Python. If you're lucky, you'll find a well-docmented example on the Internet, and can adapt it to your needs. But there are lots of frameworks that aren't well documented, or whose documentation assumes that you have some baseline knowledge in order to understand what's written.

Historically, this has meant that even an experienced developer requires a long ramp-up when they move into a new space. Which acts as both a deterrent to the developers themselves, because they might not be able to justify their current salary for lower productivity, and for hiring managers who want people to be productive immediately. Not to mention Human Resources departments that craft job postings that expect multiple years of experience in technologies that have only been in the market for a year.

I've been lucky: throughout my career I've found companies that are willing to hire me for the skills that I have in one area, but allow me to branch out into others. I think that LLMs offer something similar: a developer can bring their general experience, and quickly bootstrap into a new field with the help of the LLM. And again, I know several people who have done this.

Of course, there's a big difference between bootstrapping knowledge and simply copy-pasting what the LLM says. And I am thinking about it from the perspective of a person with over 40 years of experience in software development. An entry-level person might not be able to benefit as much, or perhaps at all.

Synthesizing Knowledge

“Knowledge” is the wrong word here, but I'm leaving it because that's the end result. What an LLM actually does is combine information that may not be obviously related. The model was trained on a large number of sources, and will identify relationships in those sources that would never be found by a human reader.

One example: I was trying to bootstrap myself into Azure Synapse Analytics, using the documentation, working through examples using data stored in Azure Blob Storage. I was able to successfully query data that I had uploaded, and wanted to create a new table to summarize that data. But every time I tried to run the query, Synapse told me that the table already existed. After several hours of re-reading documentation, I decided to ask on Stack Overflow. And got nothing back (which is not surprising, given that SO was already in its terminal decline).

A few days later I was talking about this with friends at lunch. And one, who was and is a strong LLM proponent, pasted the content of my SO question into Claude. It came back with five answers, four of which were wrong. But the fifth pointed out that my URL was missing a component, which turned out to be the answer. My friend also asked Claude to provide references, but none of them called out the actual problem; it wasn't until I re-read the documentation with the knowledge that the LLM had provided that I was able to understand the actual problem (and write an answer to my question).

Is this synthesized information always correct? Of course not! It's the job of a competent software developer to explore the information they have at hand, understand it, and be able to use it. The LLM merely gives that person a nudge toward a direction that's possibly correct. But that's much, much better than wading through a sea of information yourself, ignoring the key sources because “they can't possibly apply.”

The Apocalypse

None of what I consider “good” uses for LLMs justify trillion-dollar valuations for the companies that produce them. The companies know that as well, so they've pitched their marketing with apocalyptic overtones: these tools will make knowlege workers obsolete; if you don't use them, you will become non-competitive and go out of business; as an employer, you can slash payroll and get the same quality of work; they might destroy humanity; but they're more likely to save humanity, so we must continue building them, and everybody must use them!.

Unfortunately, these claims have fallen on receptive ears, especially the ones about slashing workforces. For a CEO, the idea of saving money by getting rid of those expensive and annoying developers (and project managers, marketeers, etc), all while increasing your own bonus in an upward-trending stock market … we've already seen the waves of layoffs.

And that increases the fear for the people whose jobs are at risk, who have also bought into the apocalypted marketing. And polarizes conversation: either you're an unethical monster, who is “slitting your own throat and those of others” or you're convinced that the LLM will lead you to a glorious (but unspecified) future. Attempting to find room in the middle will have you lambasted by both sides.

Offshoring

I can't help but compare LLM adoption to the wave of offshoring in the early 2000s. At the time, I viewed it as a backlash to the exuberance of the dot-com boom. It's much the same argument: you can lay off your full time employees and contract people who are just as good for a fraction of the price. The main difference that I see is that the offshoring wave mostly impacted companies where software was a cost center supporting some other product; the LLM wave has overtaken companies whose product is technology.

Regardless, I think that offshoring is a useful guide to how LLMs are going to be used in the future. Some companies were able to offshore successfully. They recognized that the real challenge in software development is communication: understanding what the users actually want and need. They offloaded some work to the offshore team, but kept the onshore team fully engaged with work that was interesting and challenging.

The other sort of company tried to offshore everything, and didn't much care about what happened as a result. They sent poorly-specified requests to the offshore team, with a 12 hour time difference preventing effective communication, and got absolute crap back. The onshore teams then had to fix this as best they could, knowing that there would be a new wave of crap arriving tomorrow. Morale plummeted, the “Dead Sea effect” took hold, and the skills of the onshore team decreased. Eventually, the pile of crap became so deep that it was almost impossible to work through, so the company put together a team to do a rewrite … usually involving consultants, because the full-timers were too busy shoveling their way through the current codebase.

It's not a very pleasant scenario, and I truly do feel sorry for you if you are caught in it. I'm at a point in my career where I can say that I won't baby-sit an LLM, although I might find myself involuntarily retired for doing so. I realize, however, that not every person has similar privilege.

The Dot-Com Crash

I think that most of the people who pay attention realize that we're in an investment bubble, one that is on the verge of popping. One of the more distressing signs to me, using the gold rush analogy, is that the companies making shovels are loaning the miners money to buy those shovels (full disclosure: I hold Nvidia puts). Another sign is that the rush to build data centers is held up, not just by citizen opposition, but by the inability to get the supporting components such as copper bus-bars capable of providing 100+ kilowatts of power to a single rack.

It's hard not to make a comparison to the dot-com boom of the 1990s. It's the same underlying cause: a new technology and investors pouring money into the companies that might profit from it. A big difference between now and then is that nobody felt threatened by a talking sock.

Another difference is that the S&P 500 index, the basis of many peoples' retirement accounts, is heavily weighted toward the companies participating in the AI boom. At the time of this writing, Nvidia alone is over 8% of the index and the “Magnificent Seven” technology companies account for eight of the top ten spots in the index (Alphabet, parent of Google, holds two of those spots for different share classes).

The optimistic view is that, even if those companies went to zero, the overall index would just return to its level at the start of 2024. The pessimistic view is that an earthquake in the technology sector will cause damaging aftershocks in the financial sector. Which wil then ripple out to every other sector in the economy. “It could be very bad” might turn out to be the understatement of the century.

But I don't lay that possibility at the feet of the LLMs themselves, and I'm not going to stop using them because of the possibility of economic collapse. We've already collectively bought that ticket, and we're on the ride wherever it ends up. With luck, the rentiers will be wiped out and the rest of us will eventually recover.

The Future?

The dot-com crash left us with the infrastructure and experience on which our modern web-connected lifestyle is built. Yes, we have surveillance capitalism, where our every bowel movements seems to be put into an algorithm to get us to spend money. But we also have online banking that lets us pay all of our bills in 15 minutes without going to the writing checks and going to the post office.

So I'm at least a bit optimistic about the future impact of LLMs. After the crash, after the economy sorts itself out, there will be a useful tool left behind. As someone who lived through the dot-com crash, however, I just wish we could get there without the intervening upheaval.

---

Author's note: all of the words in this post are my own. No LLMs were used at any point. And yes, I use em-dashes. A lot. You'll see them throughout my writing, including in a book that was physically published in 1992 (and which was apparently used by Anthropic to train their model).