Polyglot Programmers

I recently received an email from a well-known consulting firm that said, in essence, “you're a polyglot programmer, we're looking for those, let's talk.”

I can see where they got that impression: in the last three years I've worked with Java, JavaScript, Scala, Ruby, Python, Clojure, and SQL. In my 30+ year career, I've worked with over 15 languages. And there are a half-dozen more that I haven't used professionally. But why would a recruiter look for that?

One simple answer is that, as a consulting firm, they have to staff projects for clients with differing development environments. A person who knows multiple languages will be easier to assign. But I think there's more to it than that: “polyglot programmers” and “polyglot application” are an approach to programming that is espoused by this same firm. One that I don't agree with.

Neal Ford is often credited with coining the term polyglot programming, in a 2006 blog post. His thesis — examined in greater detail by Dean Wampler in this 2010 presentation — is that different languages have strengths in different areas, and programmers should use the best tool for the job.

I don't think that it was coincidence that this philosophy emerged in the mid-2000s, from people associated with the Java ecosystem. This was a time when Java-the-language had stagnated but the JVM was the base for a flourishing community of “not Java” languages. Groovy in 2003, Scala in 2004, and Clojure in 2007 are some of the notable examples, although there are many others. All of these languages offered things that Java did not, higher-order functions being one of the more obvious. And all of them offered the ability (more or less) to interoperate with existing Java code.

Given these new abilities, it seemed only reasonable to adopt them: to use Groovy for your XML processing, or Scala for-comprehensions to process nested structures, or rewrite your multi-threaded code to use Scala actors. And perhaps not stop there, but adopt Martin Fowler's strangler application pattern, replacing the underlying Java code entirely. And who wouldn't want to leave behind Java's verbosity and definitional boilerplate?

Ten years have passed since Neal Ford's post, and I question whether the premise is still valid (if it ever was). Here are a few reasons:

• Convergence of languages

  The mid-1960s also saw a flowering of programming languages with widely varying features: APL (1964) for mathematics, BASIC (1964) as an introductory language for education, PL/I (1965) for large-scale applications, Simula (1965) for simulating real-world interactions, SNOBOL (1962) for text processing, and many others. I was a toddler at the time, so have no first-person knowledge, but I think that these languages arose from much the same situation: mainstream languages lacked features, and the reduced cost and increased sophistication of computer systems lowered the barrier to creating something new.

  But by the end of the 1970s, most of these languages were on the path to obscurity (BASIC notwithstanding), and older languages had adopted many of the ideas that they promoted (even FORTRAN became block-structured). And then there was C, a new language that borrowed ideas from the earlier languages but recast them in a form that was better suited to the rising world of microprocessors.

  If history repeats itself, I think we're in the process of a similar consolidation. Java now has higher-order functions; if that was your reason for looking elsewhere, is it still valid? JavaScript is available outside the browser, it's performant, and now has a vast collection of libraries; is there still a reason to use separate languages for your front and back end code? Or will there be something new that takes over the world?

• Increased maintenance cost

  Many of the people in favor of writing portions of their system in alternative languages point to the efficiency with which they can write code. And while I accept the truth of that, I also realize two things. First, that writing code is a tiny part of initial implementation. And second, that initial implementation is a tiny part of the effort required during an application's lifetime.

  Given this, using multiple languages for a single application means that all of the people tasked with its maintenance have to be at least comfortable with all of those languages, if not experts. This can be particularly painful if one of the team members has a fondness for an obscure language.

  But even in the case of popular languages, it can be a problem. One of my projects was with a company whose main service was written in a mix of Node.JS and Rails. I think that their original goal was to transition from Rails to Node, but it went awry: the Rails developers left and the Node developers realized that Node didn't (at that time) have all the features they needed. So both codebases remained a core part of the business, and the company had to find people with both skillsets in order to maintain the software.

• Programmer focus

  In my opinion the best reason to be wary of polyglot projects is that developers can only know a limited number of languages — I use 1½ as my personal limit. Oh, sure, I've met people who claim to know a half-dozen or more languages. But when pressed, they only “know” those languages at a very basic level.

  True knowledge extends past syntax and semantics, to idiom and environment. It is the ability to choose the best implementation for any given goal, without thinking — the stage of learning known as “unconscious competence.” And with programming languages, the best implementation may be very different depending on the language.

  Which is not to say that you can't transition from one language to another. That's easy, I've done it several times in my career. But you'll find that the language you're most familiar with colors the way you work with the new language — when I started working with JavaScript, I attempted to use Java-style constructor functions rather than a map of data. You'll know when you've transitioned when the new language changes the way you write the old.

While I don't think that polyglot applications are a good idea, I wholeheartedly support learning multiple languages as a way to expand your abilities. A few years ago I worked my way through Programming Erlang, and consider that time well-spent: it introduced me to pattern matching, gave me a deeper understanding of actor systems, and showed me an elegant (if inefficient) way to implement Quicksort.

But it didn't make me want to mix Erlang and Java in a single project.

Practical Git: Recovering Orphaned Commits

> git reset --hard 5b7830c203e3f581c1d23d9f945478b9f94979da
HEAD is now at 5b7830c third revision

Some time later, after the window showing the log has been closed …
“Oh s**t, that was the wrong revision!”

If you've followed my advice, you have a backup of your Git repository and recovery is a simple matter of pulling the latest changes. But maybe you haven't pushed recently. Or maybe this is a scratch project that you never bothered to copy remotely.

The commits are still in your local repository, but they're orphaned: no branch head points to them, so there's no easy way to access them. But with a little digging, you can find the commit(s) that you wanted to preserve, and create a branch that contains them. You just need to know how Git stores commits.

The short form is that Git doesn't store “commits” per se, it stores objects. A commit is simply one type of object (the others are trees, which contains directory listings, and blobs, which contain files). This seems like a nit-picking differentiation, but it means that you have to hunt for your commits within a possibly large directory of objects.

That directory is .git/objects, which resides under the root of your project. If you look inside this directory, you'll see a bunch of sub-directories:

ls -l .git/objects

total 68
drwxrwxr-x 2 kgregory kgregory 4096 Apr  2 08:16 04
drwxrwxr-x 2 kgregory kgregory 4096 Apr  2 08:16 0d
drwxrwxr-x 2 kgregory kgregory 4096 Apr  2 08:16 21
…

And if you look inside one of these sub-directories, you'll see a bunch of files:

ls -l .git/objects/04

total 4
-r--r--r-- 1 kgregory kgregory 166 Apr  2 08:16 fbee2d3610317180c3f15b0d122e24f39fa82c

Well, in this case not so many files, because this is an example project with just four commits. But in a real project, you may have hundreds, if not thousands, of files in each directory. So you need some way to winnow them down. On Linux, the find command will let you see files that were created within a time range; in this case I look for everything in the last hour, because those are the commits that I deleted:

find .git/objects/ -cmin -60 -type f -ls

796801    4 -r--r--r--   1 kgregory kgregory       31 Apr  2 08:16 .git/objects/ce/06f54a5a2032d1fb605284e7217fca9e7a5073
796811    4 -r--r--r--   1 kgregory kgregory      167 Apr  2 08:16 .git/objects/66/82d6271a5416ed0a325cbafc34b32bbf893976
796803    4 -r--r--r--   1 kgregory kgregory       56 Apr  2 08:16 .git/objects/fa/b61137a51c608783b342d6e1912f45ae24c775
…

At this point it should be clear that these files are named for SHA-1 hashes. The two-level directory structure is designed so that you can store large numbers of files efficiently: in a project with 10,000 commits there will be 256 sub-directories, each of which will contain an average of 39 commits. Unfortunately for us, the full SHA1 hash consists of the directory name concatenated to the filename. So we need to apply some sed:

find .git/objects/ -cmin -60 -type f | sed -e 's/.*ects.//' | sed -e 's/\///'

ce06f54a5a2032d1fb605284e7217fca9e7a5073
6682d6271a5416ed0a325cbafc34b32bbf893976
fab61137a51c608783b342d6e1912f45ae24c775
…

The reason that we need the hashes — versus simply grepping the files — is that the files are compressed. However, git provides the cat-file tool to help us:

git cat-file -p 6682d6271a5416ed0a325cbafc34b32bbf893976

tree ec6da2f24b113700e2d64b773b4d2c9149451bfd
parent 5b7830c203e3f581c1d23d9f945478b9f94979da
author Keith Gregory  1459599418 -0400
committer Keith Gregory  1459599418 -0400

fourth revision

At this point, finding the commits that you care about is a matter of passing the list of hashes into cat-file, and grepping for text that identifies your commit. Looking for the actual commit message is chancy (especially if it might overlap with filenames or file contents), so I generally just look for files that contain “committer”:

for f in `find .git/objects/ -cmin -60 -type f | sed -e 's/.*ects.//' | sed -e 's/\///'`
do git cat-file -p $f | grep committer && echo $f
done

committer Keith Gregory  1459599387 -0400
275a303e89187e0ccffef18fd8f8d42103b33618
committer Keith Gregory  1459599402 -0400
5b7830c203e3f581c1d23d9f945478b9f94979da
committer Keith Gregory  1459599418 -0400
6682d6271a5416ed0a325cbafc34b32bbf893976
committer Keith Gregory  1459599374 -0400
0dc65221c3e1d4991fc9bf9471d5dc6e372c3885
committer Keith Gregory  1459599395 -0400
04fbee2d3610317180c3f15b0d122e24f39fa82c

The next-to-last field on each line is the timestamp of the commit; the highest number will be the last commit. At this point, you can recover your commits:

git checkout -b my_recovered_commits 6682d6271a5416ed0a325cbafc34b32bbf893976

Switched to a new branch 'my_recovered_commits'

> git log
commit 6682d6271a5416ed0a325cbafc34b32bbf893976
Author: Keith Gregory 
Date:   Sat Apr 2 08:16:58 2016 -0400

    fourth revision

commit 5b7830c203e3f581c1d23d9f945478b9f94979da
Author: Keith Gregory 
Date:   Sat Apr 2 08:16:42 2016 -0400

    third revision
…

Layoffs

Where are you in your company's layoff list? Every company has one, even if it's just a vague idea in the mind of the CEO. By looking around, you should be able to determine approximately where you stand. In my department of 20 people, for example, I believe myself to be #3. Maybe #2, perhaps as low as #5. But one of the first people to go, regardless.

Let me stop here, to stress that being laid off does not mean that you're incompetent, or less competent than the people who were retained.

Layoffs are a coldly rational response to the prospect of running out of money. Sometimes they happen well in advance of a crisis, sometimes they happen when the company is on the brink of bankruptcy. But in either case, being laid off simply means that the skills you bring to the table are perceived as being less valuable than the associated cost of your salary and benefits. I realize that's not much comfort when you're the person being laid off.

I've been laid off three times in a 30+ year career. The first time was six months into my second job, and happened on the Monday that I returned from vacation. I was working for a subsidiary of a midwestern company that made laboratory instruments, an acquisition that had been an independent startup a year or so earlier. For whatever reason, the parent company decided to consolidate operations at their home office. We were all gathered into the large conference room, and told that we no longer had jobs. The third time was similar: I was working for a subsidiary of a subsidiary of a large telecom company that learned too late that flip-phones were passé.

There's not much to be learned from these two examples, nor much to be done about them; you simply have to be prepared. If you're lucky, they come with a decent severance package.

My second layoff was more interesting. It happened toward the end of the dot-com crash. We'd been through three rounds of layoffs already, along with an enforced vacation during the winter holidays. At the start of the new year I sat down with my division VP, and said “I know I'm now at the top of the layoff list, let's talk.” Six months later (after I'd been given a large retention bonus) the fourth round hit, and he said “you were right.”

So how did I know? Like many things, your likelihood of being laid off can be expressed as a quad-chart:

The left axis is salary: the more expensive you are, the bigger a target you become. You may argue that this doesn't take into account productivity, but I believe that's irrelevant. If you're at a company that values productivity, then everyone will be productive; the idea that there's a 10x difference between top and bottom is a myth. And if you're at a company that doesn't value productivity, you're just a cog in the machine with a known cost, regardless of how productive you think you are (and really, what are you doing at such a company?).

The bottom axis, strategic versus tactical, is the one that I think is more interesting. Strategic work is for the future: shaping your next product or next release. Tactical work is about now: operations, bugfixes, responding to immediate customer desires. When a company is forced to choose, they'll value tactics above strategy every time. In the case of my second layoff, I wasn't the highest-paid person but my work was entirely focused on the future; that put me at the top of the list.

So what do you do with the knowledge that you're near the top of the layoff list?

The facile answer is that you update your resume, but that's wrong on several counts. First, of course, is that your resume should always be up to date, in case the perfect job gets dropped in your lap. Second, if you change jobs you're likely to move from the top of one layoff list to the top of another, without the offsetting benefit of tenure (yeah, it isn't a simple quad-chart; it never is). And finally, because knowing where you are on the list only matters when layoffs are imminent.

If you're in a company that's sailing along, with no storms on the horizon, there's little reason to care. Yes, there's always the chance of a black swan event that puts the company in crisis, but those aren't worth sleepless nights.

If you work for a company that does have obvious financial difficulties ahead, you should consider your options. Perhaps you transition from a strategic position to one that's more tactical. Or, again, do nothing. I work for a venture-funded startup, which means that either we become profitable or go out of business. But I'm doing the job that I was hired for, I'm enjoying it, and I'm prepared for the possibility of finding myself without a job.

And if you work for a company that has had layoffs and looks like it will have more? Perhaps then it's time for an honest conversation with your manager (or better, your manager's manager). Talk about where you stand on the layoff list, and what steps you can take to either lower your position or ease your departure.

In my case, I was already thinking about moving to Philadelphia to be with my now-wife. Perhaps my conversation with our VP raised me to the very top of the list if I wasn't already there. But it meant that both of us were prepared for what happened six months later.