adelio.org

This article is in German available here / Der Artikel ist auch in deutsch verfügbar

Every once in a while I’m faced with the question how long it will take until you become a reasonably good programmer. Even from my circle of friends and acquaintances I was already asked if they cannot help me out with a project. Here a brief try of an explanation why this is NOT incidentally possible.

My answer to this question, how long it will take to become a reasonably good programmer, is usually:

It depends. But I would roughly estimate 5 years.

With this statement the asker is usually not very content:

Come on, it can’t be that hard!

Due to the fact that I’m mainly into web development with Java, I’m going to address a few frameworks and technologies, that you are going to need on a daily basis with a brief explanation why you are needing them. For almost every item in this list there are alternatives out there but my goal was not to assemble a comprehensive list, rather I want to point out that programming is *more* than knowing a programming language.

1. The operating system. Everybody who uses a computer is using (knowingly or unknowingly) an operating system. In my case I’m using Windows as well as Linux. In general my web applications will run on a Linux system as a productive system. That means: I must know for both systems how to work with and administer them. That includes e.g. the use of the console (yes, even under Windows you are from to time to time faced with the console) or setting environment variables. Both are things where user John Doe is already overcharged. Also small differences between those systems like case-sensitivity can cause confusion in the beginning.

2. The concepts of a programming language. I’m using an object-oriented language which means that you have to understand the general concepts behind OOP. What are classes, inheritance, polymorphism? The abstract concepts behind those words are quite easy to grasp. Using them in practice in the right situation can be so cumbersome that some programmers are (sadly) never able to do in there whole life.

3. The programming language. In my case Java. The syntax of Java is not very complicated. Until you gain an oversight over the provided functionality it may take a few months. And if you never programmed with Java you need to pick up some basics first. What is source-code, what is a compiler, what are packages, classes, exceptions, data types, methods? How do I compile my source? How do I execute my generated *.class-files? How can I bundle several classes in a JAR? What the heck are JAR-files anyway? How do I include third-party-libraries?

4. The IDE. After you’ve learned the basics and you compiled and ran your first programs from the console, you definitely want to use a more sophisticated tool. You will need an IDE. It’s completely irrelevant if you are choosing IntelliJ, Eclipse or Netbeans. Personally I’m using Netbeans since over six years and I’m still discovering sometimes Features that are new to me. An IDE IS a very powerful tool, build for professional developers. Until you learned how to use this tool properly a lot of time will pass.
As an alternative an editor with syntax-highlighting could be already a progress. But the integration with all the developer-tools that you need on a daily basis is really something very charming.

5. The build tool. At the latest when you start working with other people on a project you want to use a build tool which can create your project independently from your IDE / your computer. In the Java world the most common tools for this are Ant and Maven. Maven comes along with a dependency-management which is indispensable (even if you wish sometimes) and for Ant you could use the Ivy project for this. And an own repository-manager like Nexus is in some situations also mandatory.

6. XML. Not every project will use XML. But for most configurations you are not getting around XML (e.g. using Maven or Ant and most other third-party-libraries).

7. A debugger. No program is bug-free from the beginning. And sometimes starring at the code is not sufficient. You will need to learn what a debugger is and how it works. Where and when do you have to set breakpoints, how to display variable values, how to step through a program and/or when/how to skip methods. Conditional breakpoints are also a nice thing. And even if you don’t need all of that from the beginning, it’s nice to know that you have the right tool for it, when you encountering a problem where one of those points comes in handy.

8. The database. Almost every program saves some data in some kind of form. Sometimes a normal text file is sufficient, sometimes a XML-document would be the more appropriate solution. But if the amount of data increases you won’t get around a database. Quite often you choose a RDBMS. But maybe for your case a database from the NoSQL-movement is more appropriate. And other times (e.g. for testing) you only need an in-memory-database. MySQL was for my scenarios usually sufficient.
Ah, and by the way: if you are using a database you should also have a bit knowledge about database design.

9. SQL. SQL is the language that you use for database. If you want to select / alter data in your database you need at least a basic knowledge of SQL.

10. An ORM. An object-relational-mapper handles most interaction with your database and reduces the amount of ‘direct’ work with the database. You will still need a basic knowledge of SQL and ORMs will bring there own pitfalls. Hibernate has become the de-facto standard in the Java-world. In Hibernate you can use for your database queries HQL, a language similar to SQL. But nowadays you should consider using JPA which uses JPQL, where Hibernate is an implementation of JPA. Confused? Good, else I would be out of work ;-)

11. The server. For an web-application you will need a server. For my applications I just needed up till now only a servlet container like Tomcat or Jetty. A complete application server like Glassfish, JBoss or Geronimo was not needed by me, yet. Also the integration of a servlet container with a webserver like Apache HTTPD was not needed yet (or was usually done by someone else).

12. The web-framework. You don’t want to reinvent the wheel every time that’s why you are using a framework which provides establish solutions to known repeating problems. I tried and used several frameworks for several projects (Struts 1, Struts 2, Spring MVC, GWT, Stripes, Wicket) and favor at the moment Wicket. If you prefer a component-oriented framework (like Wicket, GWT or JSF) or a page-oriented framework (like Stripes, Struts, Spring MVC) is a question of taste.
But for all of the mentioned frameworks you will need a basic understanding of the technology that drives the web. What is HTTP, what’s a POST-request, what’s a GET-request, what are parameters, what is a port and how is a session working? What is a server roundtrip? And why is AJAX a nice gimmick for the end-user but for developers sometimes just hell?
And before I forget it: of course you will need a bit of HTML and CSS. And for the AJAX-stuff you want to have a bit knowledge about JavaScript. And while we are at it: the first time you are seeing strange characters on your page (while using Umlauts or special characters) you will have the joy to learn about encodings. (Why the heck is there not only UTF-8 out there?!?)

13. Securing a web-application. Most websites are having a secured area where a login is needed. You don’t want to implement this on your own. Choose rather something like Spring Security.

14. Logging. As soon as a customer reports an error you will be glad when you were protocoling when, what, how something happened in you code. So you are gonna need a logging framework like Log4J.

15. Dependency Injection. To couple all those technologies in a loose way you want to use a dependency framework like the de-facto standard Spring. Latest if you need to mock objects during testing you are gonna love Spring.

16. Unit testing. You don’t want to test your complete application manually after every change just to see if your changes broke something. Rather you will write unit-test which will test your code programmatically. If you are using TestNG or JUnit is a matter of taste. I prefer TestNG even though JUnit is more widely spread.

17. A version control system (=VCS). Due to the fact that you usually never really finish a software project, instead you just stop working on it, it might happen that you will end up with several versions. And as soon as more than one developer is working on the project you want to have the possibility to track changes in the source code. That’s what version control systems are for. Here you also have the agony of choice: CVS, Subversion, GIT, Mercurial, etc. If you should choose Subversion as your first version control system you can’t do much wrong, especially because it’s integrated in most IDEs.

18. Continuous Integration (=CI). Unit-tests are taking time and need to be executed on a regular basis and sometimes you just don’t trust the changes your co-workers committed. Here a CI-server, like Hudson can be very handy. The CI-Server checks your project out of the VCS in a regular interval, compiles and test it and do whatever else you tell him to do. What else can be done we will see in our next item.

19. Static code analyzers. Some potential bugs can be found by tools, so called static code analyzers. Those tools can be integrated in your CI-process. Findbugs, PMD, CPD, Checkstyle or JavaNCSS are just a few which I use on a regular basis. But keep in mind that you need some experience to interpret the results, especially to identify false positives.

20. Bugtracking. Sometimes you can’t keep up with bug fixing so that you should set up and use a bugtracker. I’m using Redmine because it offers also an integrated wiki, forum and a SVN-integration. Other famous bugtrackers are Mantis, Bugzilla and JIRA. Oh yeah, of course you have to have the right infrastructure to use those tools. Redmine uses Ruby, Mantis uses PHP and Bugzilla need Perl. Your choice :-)
Another criteria you should keep in mind before choosing, is, if you need an integration in your IDE.

21. Profiling. If your code is not fast enough and everything in your program takes an eternity to finish, you will use a profiler to spot the bottlenecks. To interpret the results and/or resolve issues that you found, you will need to some extent knowledge about the technical internals.

22. Memory Management. In Java you don’t have to handle memory management on your own anymore. Nevertheless you should know that there is something called garbage collector and you should also know more or less how it works. When you are developing web applications you will run sooner or later into an OutOfMemoryException which is caused by a small application, even though you have X gigabyte RAM. To understand the root of this error and how you can solve it, you need to know what is going on under the hood.

23. Algorithms. There are more ways of killing a dog than by hanging. A lot of problems can be solved in a several different ways. With different advantages and disadvantages. Just have a look at the variety of search algorithms. Sometimes you will have a working solution but need to rewrite that part because of changing parameters which makes your first solution not optimal anymore. Changing working code is also called refactoring.

24. Refactoring. Nowadays with modern IDEs refactoring is almost like shooting fish in a barrel. The goal is to make the code more maintainable or in general “better”. You just need to know what “better” is. For the most common problems there are common solutions which are called design patterns.

25. Design Patterns. Those are solutions for common problems. To make use of them you need to know respectively realize that you are encountering a certain problem.

26. Programming Style. Even though formatting your source code has no influence on the flow of your program (at least not in Java), a good programming style is vital for maintainable code. Your style will get (hopefully) better the more experience you gain.

27. Documentation. Creating a good documentation is *much* harder then it seems at the first glance. It will take some years until you will have a feeling for what is important enough to be documented and what not and how to structure a good documentation.

28. The English language. You won’t get around the English language as a programmer. If you just need to read an English text you can maybe conceal your ignorance. But if you need to write in English – and you will as a programmer – you should make sure that your writings don’t get too adventurous.

29. Other matters. A few buzzwords which pops into my mind when I think about what else you might need: Lucene for searching, Hibernate Search to integrate Lucene with Hibernate, SSH to work on a remote machine, Firebug for finding bugs while developing websites, Google Page Speed to optimize speed, Java Mail for sending mails (this implies again that you need to know what’s working under the hood (e.g. what is a SMTP-server?)). Quartz for cron jobs, regular expressions, webservices (if SOAP or REST doesn’t matter), a general understanding of computer security (nobody wants that his application is e.g. vulnerable for SQL-injections). Multithreading needs to be mentioned nowadays and also deadlocks and heisenbugs. Oh, and I almost forgot about all the different licences that you will encounter if you want to make use of third-party libraries.

30. Last but not least: an unlimited frustration tolerance, an eye for details and the patience of a saint. Everybody who stared for a whole week at only three single lines which caused a deadlock and couldn’t come up with a solution knows exactly what I’m talking about.

Summary
Do you need all this stuff? It depends! If you just want to create a little guestbook then definitely not. But I just presented a set of tools here. If you want to beautify your house and you want to hang up a picture in it, it’s sufficient to know how a hammer works. But if you want to build your house from scratch you will need a lot more tools. And a lot of them in different designs.

Software development is a bit like driving a car. It’s quite simple to get your driver’s licence. But no new driver will dare to start his first 500 miles trip in a rainy winter night all alone. It’s possible. But I guess no one would do it or would even have the confidence to do so. Programming beginners are a bit different. They read the first half of “Learning XYZ in 21 days” and think they are capable to start the next Microsoft.
I also met students that took for one semester a programming course and they really believed that they would know everything now.

This list is neither complete nor is it exaggerated. Who believes that I’ve exaggerated should give it a try to build a piece of software over a few months. All mentioned tools are free of charge except for Windows and JIRA.

And to all readers who made it till here and know a bit about programming: feedback is welcome!