Kategoriarkiv: Systemutveckling

Memoization

This blog post is also available on the Fortnox developer blog.

As developers we have a large selection of design patterns and techniques at our disposal when we design and implement our solutions. This is especially true in object oriented code; patterns such as decorators, builders and factory methods are just some of the patterns you probably see during your workday if you are coding with that paradigm.

Most of these design patterns have names that reveal their purpose. If we take the three patterns mentioned in the previous paragraph, the decorator pattern decorates an object with additional behaviour. The purpose of the builder and factory patterns are also reflected in their names; the builder pattern lets you construct an instance of a class using a fluent interface, while the factory method pattern is used to separate the construction of a complex object so that it resides in its own distinct method.

But then there are other techniques whose names do not convey their purpose as clearly. One example I would like to mention here is memoization, which is a concept I have stumbled upon several times during my days and figured that it must be something very mysterious. With a name like that it must be something really hard to understand, right?

Well not really. Memoization is basically an optimization technique in which a cache is used to store previous results of expensive computations. If the computation has occurred once, the following calls to the same routine will return the cached value instead of redoing all that work again.

In my attempt to understand how the technique works I decided to implement memoization in JavaScript, and this implementation can be seen below. The code defines a function called memoization, which closes over an initial numeric value. This value will then be used as the first operand for simple addition calculations. When you call memoization with a numeric value, that value will be used as the other operand and the result is returned. Previous results are cached according to the memoization technique.

let memoization = (val) => {
  let cache = {};

  return (other) => {
    if (!cache[other]) {
      // replace assignment below with your expensive computation
      cache[other] = val + other;
      console.log('Cached new value');
    }

    return cache[other];
  }
}

let plusFive = memoization(5);

// first call caches the result and outputs "Cached new value".
plusFive(10);

// second call reuses the cached result, i.e. no console output this time.
plusFive(10);

Now you’re probably thinking that a simple addition is not a very costly computation, and that’s true. The whole idea of this example was just to show how an implementation of the technique can look like. Apparently, the technique is useful for purposes other than optimization, but that is for you to dig into if you want to know more. :)

That’s all for now. Until next time!

Learning JavaScript properly

This blog post is also available on the Fortnox developer blog.

During the early 00s my programming was more or less about creating and maintaining web sites in LAMP setups. From that period I remember fighting many battles trying to make JavaScript code work across web browsers. It was not fun. For many years now I’ve kept my distance from the language, hoping for a better replacement to arrive at some point.

The thing is that JavaScript survived and is used extensively in all kinds of web setups, which is why I’ve decided to give the language another go. So I picked up JavaScript: The Good Parts (that have collected dust in a book shelf at home for several years) and saw this really nice presentation on functions in JavaScript recommended to me by a colleague at work. I read up on higher-order functions in the language and learned about some new stuff like arrow functions and constants (which are pretty cool). And then I decided to program something based on my understanding gained from all those resources.

My idea was to avoid the prototype stuff that I haven’t grasped fully yet and instead build something using functions and closures only. In the end I chose to build a simple snake game using JavaScript and React. Using React for a snake game is maybe overkill, but I found that the Create React App project gave me a nice bootstrapped environment for me to work in without the need to setup Babel, Webpack and all those other things you need to worry about. A few days in now there is a working version of the game that you can try here if you want. Just use the arrow keys or swipe to control the snake. Although the code is most probably far from idiomatic JavaScript, I decided to publish the code on GitHub, with the plan to ask some colleagues more experienced in the language to tell me what I’m doing wrong.

From this basic snake implementation I take with me a few insights. First, my time with the language was actually more pleasant than I had imagined. I believe that programming in Clojure and other functional programming languages have made me let go of the object-oriented paradigm a bit. This time around, when I decided to stick with using just functions and closures, the code felt more natural to write than when I tried to program like I do in Java.

Secondly, I miss working with persistent data structures and having robust ways to manage state changes. The ability to stay immutable takes away a lot of complexity not really related to the problem you want to solve and lets you instead focus on what’s important. You can get all this by simply using ClojureScript instead of JavaScript (as ClojureScript compiles to JavaScript) but I’ll save that subject for another post. 😉

A final and fun takeaway from this exercise is that the snake in a snake game can be modeled as a queue (along with a variable to control its growth when it feeds). Let’s have a look at how this works, starting with the algorithm to use each time the snake moves.

1. Push the next coordinate to the queue.
2a. If the grow variable is zero, then pop the first item from the queue.
2b. Else, decrease the grow variable by one.

Consider the following queue with x and y coordinates represented as tuples. JavaScript lacks both a tuple and a queue data type, so good old arrays will have to do.

let grow = 0;
const queue = [[1,1], [2,1], [3,1]];

So above we have a queue with three coordinates representing the snake. In this example, the snake is facing right, which means that the front of the snake is at [3,1] and the end of the snake is at [1,1]. Let’s move the snake one step to the right. According to the above algorithm, we push the next coordinate to the queue, which means that the queue now looks like this:

queue.push([4,1]);
// queue is now: [[1,1], [2,1], [3,1], [4,1]]

The grow variable is zero, so we pop off the first element like so::

queue.splice();
// queue is now: [[2,1], [3,1], [4,1]]

Alright, so now we have moved one step to the right! Let’s pretend that the snake found something to eat at [4,1], which means that the snake will grow. In the current version of the game, the snake grows by three whenever it eats, so we’ll increase the grow variable by three. Now, let’s move the snake to the right again. We first add [5,1] to the queue. This time, the grow variable is greater than zero, so we will not pop an item off the queue this time but rather reduce the grow variable by one. The outcome of the first movement we  just described is shown below.

queue.push([5,1]);
grow--;

// grow is now: 2
// queue is now: [[2,1], [3,1], [4,1], [5,1]]

As the snake keeps on moving the grow variable will eventually be reduced to zero. When that happens, we will start popping items off the queue again whenever the snake moves. I hope you found the snake discussion super interesting and life changing. 😉 That’s all for today, until next time!

Introducing Clojure

This blog post is also available on the Fortnox developer blog.

At work we have something called the ”Java forum” where we discuss Java and related technologies from time to time. A while ago I felt that I wanted to contribute with a beginners talk about Clojure for programmers. I’m no expert in the language but I know enough to present at least the fundamentals. So I signed up for a talk and went home to prepare a presentation.

What I came to realize in front of the computer was that it is pretty hard to present Clojure in like 20-30 minutes. To generalize a bit, I would say it is practically impossible to give a proper introduction to any language in that limited time. Clojure is particularly tricky in this sense; a LISP dialect with persistent data structures is pretty far from what Java or JavaScript programmers easily relate to.

I ended up changing my presentation material over and over again, excluding and including things back and forth, but in the end I think it turned out alright. I had to leave out sets, atoms and other cool stuff as there simply was no time to go wide or deep. Anyway, I figured that someone else out there might be interested in an example ”agenda” of how to introduce Clojure to developers in such a limited time frame, which is why I uploaded the material to a GitHub repo here. Each expression in the clj-file comes with comments and suggestions of what to say when evaluating them. I used the Light Table code editor during my presentation and that worked pretty well. With that editor you get a nice inline evaluation of your expressions, which is really handy while presenting.

I hope someone out there finds the material valuable in some way or another. Until next time!

A troublesome query

This blog post is also available on the Fortnox developer blog.

Last week we decided to have a look at one of our database queries that had started to become a real problem. The symptoms were super strange; occasionally during peak hours the query began to perform really, really bad, from around 200 milliseconds per query up to about 30000 milliseconds per query. Yes you read that right, we are talking about half a minute or at times even longer than normal. That’s like 150 times slower and of course not acceptable at all, so we went head in trying to figure out why. This blog post describe our findings and how we found a solution to the problem.

The first thing we did was to set up a local environment to try to recreate the problem there. We use Postgres and Java in our application, so we decided to recreate the problem in a test harness using JUnit with the same database driver as we use in production. After setting the connection pool size to 1 and repeatedly firing away the problematic query we were able to recreate the behavior quite easily. The symptoms were really weird; the first nine queries were perfectly fine performance-wise, but from the tenth query and onward the query started to take up to 150 times longer. What’s going on?

After tweaking the query, altering the prepared statement parameters back and forth and trying to understand the Postgres server logs, we finally started to grasp what was happening under the hood. The key breakthrough was probably when we came across this information concerning server-side prepared statements. It turns out that there is a special threshold that determines when server-side prepared statements should kick in. This threshold has a default value equal to 5, which means that the statement should be prepared ”often” on the server.

When we increased this threshold value a bit (to 10), the bad performance of the query instead occurred around the fifteenth time rather than on the tenth time. According to the PGStatement javadoc, setting the threshold value to 0 effectively turns off server-side prepared statements. We gave that a try and instantly the problem completely disappeared. Interestingly, the performance in our initial tests were not affected whatsoever. The query performed just as well using only client-side prepared statements. The poor performance of that query vanished completely and has not caused any problems in production since we turned server-side prepared statements off.

Server-side prepared statements might be a killer feature for some queries, but for the one we were struggling with it definitely was not. A very interesting finding we take from all this is a line of text from the PostgreSQL Extensions JDBC API documentation. It goes like this:

”You should be cautious about enabling the use of server side prepared statements globally.”

Okay, but it’s not like we enabled them globally; the default threshold value is 5, which means that they are enabled by default if you do not explicitly turn server-side prepared statements off. If the recommendation is to not enable them globally, a default value of 0 would in my mind have been more reasonable, right? Hopefully there’s a good reason for its current default value that someone more into databases than me has the answer to. In any case, as demonstrated in the API documentation referred to earlier, it’s quite easy to set the default value via the connection string, like so:

// 0 means that we do not use server-side prepared queries
String url = "jdbc:postgresql://localhost:5432/test?prepareThreshold=0";

I hope this blog post can help remedy some headache out there. Until next time!

The best programming language

This blog post is also available on the Fortnox developer blog.

A couple of weeks ago a friend of mine posted a very open (and admittedly pretty bad) question on #VXODEV, the local Slack developer community here in Växjö. The question can roughly be translated into something like ”Which [programming] language is the best?”. Naturally he and the members in the channel knew that there is no such thing as the ”best” programming language. Instead, the question was asked to get a discussion going on why people prefer one language over another. From my perspective as a novice Clojure enthusiast I had to stop and think for a moment. Why do I keep picking up Clojure for my late night hacks over other languages?

It’s not like Clojure helps me to solve problems that cannot be solved with other programming languages. Sure, it is true that a particular technology may be more performant or better suited for a particular task than others, but in most cases it doesn’t really matter.

Take for instance the Advent of Code (AoC) challenges that I’ve been having fun with lately. In these challenges you are free to choose whatever technology you want to use when solving the puzzles. Wouldn’t it then make more sense to just pick up the programming language you feel most comfortable and productive with? My main language as of now is with no doubt Java. I have been working as a Java developer for many years now, and I am pretty sure that I would solve challenges in AoC a lot faster with that language than with Clojure. Despite this I’ve not considered using Java in any of the challenges as of yet.

Perhaps the programming paradigm supported by the language has something to do with it? Clojure is a functional programming language, and the declarative style of programming that comes with that paradigm is something I’ve grown quite fond of.  The thing is that nowadays Java also has support for functional programming; since version 8 of the Java language developers can apply functional programming in their code. And while Clojure is predominantly a functional programming language, other languages that fall into a similar paradigm category, e.g. Haskell, pretty much lie in my toolbox collecting dust. Which in a way is kind of sad as Haskell is cool.

At the time of writing this blog post I looked through the history of the Slack channel to check what my answer to the ”which {programming] language is the best” question. It read something like the following: ”I haven’t come up with an answer yet, but what I do know is that I’m having a great time writing Clojure code”. And I think that ”a great time” is a pretty sound argument here. I mean, if you spend time in the evenings writing code it should be fun, right?

But what is fun for you may not be fun for me. For instance, a TV show may have some qualities that you enjoy but I don’t appreciate as much. Some of those can be quite obvious and be put into words, e.g. ”I don’t like sci-fi”. Other qualities are more or less based on feelings that are not as straightforward to identify and express. They just feel right and makes things interesting enough to make you watch the entire season of the show. It’s like, ”hey, this hammer feels nice in my hand so I choose it over the other ones in my toolbox”, but there is no single and obvious explanation why.

With the above ramblings in mind, I guess the ”best” programming language (for you) is a language that through some desired qualities makes you enjoy the creative process of programming.

But then again, there is no such thing as the best programming language (at least in the general sense). 😉

I’ll end this little text with a related blog post about how programming languages may relate to different personalities. Enjoy!

Until next time!