Robert Važan

What's in a good software library?

High-level advice on API design from author of several opensource libraries and a seasoned developer.

Programming can be thought of as a form of communication with computers. There's even an AI theory that says that intelligent computers are merely ordinary computers that can be programmed quickly and easily. So what makes computers easy to program? Or in other words, what makes programmers productive?

If programming is communication, then libraries provide vocabulary for this communication. Relationship between programming languages and libraries is like relationship between human brain and knowledge. Libraries and the vocabulary they provide (akin to knowledge) matter more than computing power (akin to IQ in humans).

Libraries have limits though. It's hard to deliver highly available database in the form of a library. Cloud services make libraries smarter and they reduce the amount of complexity visible to programmers.

People often think of programming as telling computers what to do. But programming is a two-way communication. Visibility matters. Black box tools are hard to debug. Specificity of communication matters too. Reporting "bug at line 333" is way more useful than reporting "bug in the program". Libraries and cloud services need their own inspection, debugging, and profiling tools. They need to speak and they need to speak clearly.

Simplicity and ease of use merely accelerate adoption of new services without making them fundamentally more useful. Accelerated adoption is nevertheless very useful in the rapidly changing software landscape.

Interactivity of communication is important. It permits experimentation. Programming can be thought of as a dialogue. It is however still important to keep tools predictable. No REPL loop can match the speed of internalized mental models. Not to mention that mental models permit partial simulation of the system while interactive shells require concrete implementation.

There is still one elusive issue in library/service design. Some tools have an obvious next step at every moment while others cause writer's block.

Scaffolding needs to be done either in mind or on paper. Such tasks show very little progress on code level while the scaffolding is being built. Once the problem is solved using the scaffolding, the code rapidly materializes.

We might be hitting essential complexity here. We need to complete complex tasks that cannot be abstracted away, because the purpose of the code under development is to provide that abstraction. There's no universal problem solver. Tools cannot be simply provided with a goal and expected to find a path towards the goal.

But there's still something tools can do. It's not fair to shut down all services the tool can provide simply because we have an incomplete solution. Tools should continue to work with the partial solution in order to encourage rapid prototyping of increasingly functional solutions.