Originally from  http://wrttn.in/04af1a (not available)

Institutional memory comes in two forms: people and documentation. People remember how things work and why. Sometimes they write it down and store that information somewhere. Institutional amnesia works similarly. The people leave and the documents disappear, rot, or just become forgotten (as it were).

I worked for several decades at a large petrochemical company. In the early 1980s, we designed and built a plant that refines some hydrocarbon type stuff into other hydrocarbon type stuff. Over the next thirty years, institutional memory of this plant faded to a dim recollection. Oh, it still operates, and still makes money for the firm. Day to day maintenance is performed, and the skilled local crew is familiar with the controls, valves, safety systems, and other such.

But the company has forgotten how it really works.

A few things conspired to make this happen:

The downturn in the oil industry through the 1980s and 1990s caused a moratorium on new hires. By the late 1990s, our group was a mix of people over 55 and under 35, with few in between.

We gradually made the move to fully computer-based design.

A series of group reorganizations physically moved our office several times.

A major corporate merger several years after that completely dissolved us into a larger petrochemical firm, causing a significant institutional and personnel shakeup.

Institutional archaeology

In the early 2000s, several of my colleagues and I retired.

In the late 2000s, the company remembered that this plant existed, and thought about doing something with it. Specifically, increase output by debottlenecking one unit, and doing a feasibility study on addition of a second unit.

Now they had a problem. How was it built? Why was it built like that? How does it work?

Institutional memory grows hazy at this point. The alien machinery hums along, producing polymers. The company knows how to service it, but isn’t quite sure what arcane magic was employed in its construction. In fact nobody is even sure how to start investigating.

It falls to some of the then-younger engineers, now the senior cohort, to dig up documentation. This is less like institutional memory and more like institutional archaeology. Nobody has any idea what documentation exists on this plant, if any, and if it exists, where it is, or what form it might take. It was designed by a group that no longer exists, in a company that has since merged, in an office that has been closed, using non-digital methods that are no longer employed.

The first step is finding out what the plant’s name is. It turns out that the name most engineers use is just a colloquial name based on its location, and it has another official name. Several of them, even. There is the name of the internal project that designed it, and the name of the joint venture under which it was actually built.

There was a unique ID assigned in 1998 as part of a document-management revamp. There is another unique ID, assigned in 2001 for digitization purposes. It’s not entirely clear which document management systems are current, incidentally. Also, some of them point to other document-management systems.

No luck here. The 1998 ID points to documents located in a “library” at an address that hasn’t existed since long before 1998, which might explain why that 2001 ID doesn’t point to any digitized documents older than some recent reports on routine maintenance. At the time, I had naively hoped digitization would solve our problems forever. My manager was reading a dense book about it that I picked up out of curiosity. It had seemed persuasive.

But, the old-fashioned phone and email tree worked a bit better. The old research division is still mostly intact, and their physical library exists. Someone there is able to find documentation on the plant’s polymer processes, as well as copies of some engineering documents duplicated for the R&D library’s local records. Big paper blueprints and engineering drawings, as well as books of data, in dusty filing cabinets. The paper documents tauntingly sport IDs announcing that they had been digitized by Big Digitization Corp at some point in the past. Who knows what happened to that archive.

Deciphering documentation

Some documents assembled, the engineers get to work trying to get a handle on how to organize a debottlenecking project. Unfortunately, the documents seem to be written partially in hieroglyphics, and are only partly complete. They make some very slow progress. The manager half-jokes that engineering schools should teach a course in engineering archaeology, where students are given a pile of 30-year old documents and asked to figure out what’s going on. I like the idea. Maybe even read an old engineering textbook, like the collectors into repairing old vacuum-tube electronics do.

Some of the methods and notation are familiar, but others are long obsolete. Even where nothing has officially changed, cultural assumptions about what should be documented explicitly or can be assumed have changed, making interpretation difficult. And it would really be nice to have a big-picture overview book. At the end of the project someone should’ve been commissioned to write a book, “What This Goddamn Plant Is: And, How It Works”. That book is effectively being written now, only by archaeologists.

Reverse corporate espionage

A former colleague and I were contacted some time after this by another former colleague who now had some sort of management role in this group. Would we be amenable to consulting part time on a project relating to the old Plantname? I agreed. It sounded interesting, and I was being offered an hourly rate amounting to several times my previous salary.

Thus I landed the strange job of trying to explain to the company how its plant worked.

I could draw on several kinds of personal memory for this job. I remembered how some things worked, and the 30-year-ancient engineering practices were my own. More importantly, I had an idea of what was important and how the pieces fit together.

Perhaps equally importantly, I unofficially had some documentation. During our office moves and reorganizations, the document situation became increasingly dire. I would wait days to get something mailed to me, after tracking down a series of merged document libraries, some of which were halfway through the digitization processes. Paranoid corporate management also had rules about anything relating to trade secrets, which meant anything relating to the polymer process at all, which made it hard to work while visiting contractors’ offices.

So, we developed a don’t-ask/don’t-tell policy of making private copies of documents and carrying them around with us. Engineers, to generalize, hate waiting around for stupid reasons, and having documents meant that we could get to work. It also made us look better, since we got things done on time, instead of having to send out lame excuses that we’re late because we’re waiting on a fax.

My job now was to smuggle these documents back into the company. I would be happy to just hand them over. But that doesn’t make any sense to the company. The company officially has these documents (digitally managed!), and officially I don’t. In reality, the situation is the reverse, but who wants to hear that? God knows what official process would let me fix that.

No, the documents need to be brought back in to where they ‘already were’ unofficially. Physical copies are made and added to the local group library. Eventually they’ll probably work their way into the digital document management system, the next time someone canvasses and notices some documents with no inventory control tags. I hope they aren’t lost this time, because I won’t be around in another 30 years to smuggle them back in again.

Oh, and as an external consultant, I’m not allowed to know some of the trade secrets in the documents. The internal side of the team needs to handle the sensitive process information, and be careful about how that information crosses boundaries when talking to the external consultants. Unfortunately, the internal team doesn’t know what the secrets are, while I do. I even invented a few of them, and have my name on some related patents. Nonetheless, I need to smuggle these trade secrets back into the company, so that the internal side can handle them. They just have to make sure they don’t accidentally repeat them back to me.

We hear a lot about the spy-movie kind of corporate espionage. I’d love to read a study of reverse corporate espionage, where companies forget their own secrets and employees have to unofficially get them back. I’m convinced it happens more than you’d think.

A solvable problem?

I’m not sure what the moral to this story is.

Better organization and document management could solve some of the problems. But attempts to fix corporate document management also caused some of them, so one has to be careful. We might’ve had better luck if more of the physical office libraries still existed. We only retained some of the documents because one of them did.

Memory of techniques and importance is even harder. Maintaining a continuous gradient of ages in the company probably helps, so you don’t fall off a memory cliff when one cohort of employees retires.

But maybe engineering archaeology will always exist. The more I look around, the more the engineering world, once you go back more than a few years, looks like subterranean New York City. A mass of strange engineering feats humming away out of sight, produced by long-forgotten ancient peoples, leaving only fragmentary maps and diagrams.

-An engineer, 2011-12-04