Category Archives: Writing Software

04.12.13
by

Do Not Flounder (Stay Un-bored)

The following is an article that I am working on for a yet-to-be-determined publication. Having done this before, I will say that getting an article published in a journal/magazine isn’t as difficult as one may think (as long as you have something to say). This hasn’t been proofed, so please forgive any typos or errors. This article is not about the role of management. I would like to follow up on that subject, because I do think management can and should help to produce great software engineers. I’ve seen otherwise good programmers flounder under lacking management. It happens. This article is about the role of the developer, the individual contributor, in making sure that his or her career starts of right and continues to grow.
One ore note: The original version of this post was written in about 2 hours and was full of errors. I’ve applied a number of corrections, but it will likely be further edited before publication.

I have no idea whether or not most developers using Agile have actually read the “Agile Manifesto.” Here it is:

We are uncovering better ways of developing
software by doing it and helping others do it.
Through this work we have come to value:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
That is, while there is value in the items on
the right, we value the items on the left more.

This post is more about career growth than Agile, but somehow I think the Agile approach is relevant here. Agile, as we know it, is an approach to software design. It can also be an approach to managing one’s own career.

Floundering

A friend of mine recently said this: “Your attitude determines your altitude.” Although he was speaking in a general sense, I couldn’t help but think of the application of this motivational advice in relation to software engineering. It is relevant because as software engineers, our career growth is in our hands—perhaps to a greater degree than in any other field. Continue reading →

04.11.13
by

Java “Losing its Mojo?” I Think Not!

Wired has an article titled Is Java Losing Its Mojo? While the article seems to contradict itself in some ways, I have to take issue with the general theme. As someone who pays attention, I simply haven’t seen this happening. On the contrary–It seems to me that Java continues to grow. Just peruse the job boards. Continue reading →

01.27.13
by

Success as a Technical Lead – Article

I stumbled upon this article today. Its a little dated (from 2008), but still relevant. I don’t think the list is comprehensive, and I certainly think other technical leads would have varying opinions on things. All of the points listed are good, but there are a some points that really stand out:

6. Be part in the design of everything
This does not mean do the whole design. You want to empower team members. But your job is to understand and influence each significant subsystem in order to maintain architectural integrity.

7. Get your hands dirty and code
Yes you should take parts of the code and implement them. Even the least glamorous parts. This will help you not getting stuck alone between management and the team. It will also help you gain respect in the team.

20. Don’t blame anybody publicly for anything
In fact as a tech lead you cannot blame anybody but yourself for anything. The moment you blame a team member in public is the moment when the team starts to die. Internal problems have to be solved internally and preferably privately.

24. Mentor people
It is your job to raise the education level of your team. By doing this you can build relationships at a more personal level and this will gel the team faster and harder. It is very effective with more junior people in the team but there are ways to approach even more senior members, just try not to behave like a teacher.

25. Listen to and learn from people
Even if you are the most experienced developer on the team you can always find new things and learn form others. Nobody is a specialist in everything and some of your team members can be domain specialists who can teach you a lot.

28. Be sure you get requirements and not architecture/design masked as requirements
Sometimes business people fancy themselves architects, sometimes they just speak in examples. They can ask for technology XYZ to be used when what they really mean is they want some degree of scalability. Be sure to avoid hurting feelings but be firm and re-word everything that seems like implied architecture. Get real requirements. To be able to do this you have to understand the business.

36. React to surprises with calm and with documented answers
Never get carried away with refuses or promises when confronted with surprises. Ask for time to think and document/justify your answers. It will make you look better and it will get you out of ugly situations.

A theme throughout the list, and throughout a number of similar books and articles with such advice, is that a good technical lead appreciates and values the various talent and particular skills of the team. A great technical leader isn’t necessarily the “know it all” of the group. He or she should certainly be skilled and eager to maintain that skill–and even be a great developer. But smartest person in the room? Maybe. Maybe not. Personally, I like working around people who are smarter than me. This is the best way to learn.

And there’s a flip side to number 20: Don’t blame people publicly for problems, but be quick to praise people for successes, major and minor. A sense of recognition for one’s diligence is tremendous motivator. I don’t know a single person who doesn’t appreciate kudos. Most parents realize that their children respond better to positive reinforcement than negative… This doesn’t change when one reaches a certain age. I’m not suggesting that a team member not be confronted for problems. Of course he or she should (and must).

Very recently a company-wide email spoke of a major success of mine (successful deployment of a year long project), and mentioned me by name. It felt great, and it made me want to continue with even more success (and it was a great confidence boost). Simple put, its good to know that the folks at the top of the organization are aware and appreciative of the work of those in the trenches!

This all may sound like a lot of feel-good fluff. It isn’t.

Little Tutorials: 36 Steps to Success as a Technical Lead

10.17.12
by

Version Control/Wiki Control

I FIRMLY believe that documents related to a project should be managed in the same version repository as the source code. This gives us a snapshot in time of all items related to a project. The problem with this comes when using a wiki (and I love using wikis, so don’t get me wrong). There is no way, if we are using a wiki page for specs, requirements, etc., to link a wiki instance in time to a Subversion (Git, Mercurial, whatever) instance in time.

And I don’t think we would want such a feature anyway. A wiki covers many projects and many team needs, not just the needs of a small group of programmers on a single project. I can’t imagine “rolling back” an entire wiki to a given snapshot.

I wonder if there are any clever ideas out there for handling a need such as this. I can see the possibility of pre-commit hooks being used to label wiki pages, but this seems cumbersome (if not entirely unmanageable). The other solution is to rely on the wiki only for collaboration and not for official documentation of any kind. This approach, unfortunately, cripples much of the power of using a wiki.

I am open to ideas.

08.07.12
by

6 Developers, One Room

Under an extremely tight deadline one team member decided that it would be best if the developers took over a conference room. On a long conference room table there are 6 computers, and 6 extremely talented developers chat, joke, brainstorm and work away. The manager of the team is there too, explaining requirements and helping to clarify definitions and functionality.

There is pizza, too.

Its like that scene in Apollo 13 where the engineers have to figure out how to get the Apollo back to Earth. Ideas bounce freely and communication is immediate. I don’t have to wait for a response to an email or a response in a chat window (which may or may not come). And there’s something about sharing a space with a common goal: The team seems to gel. There is little or no arguing or passive-aggressive commentary as I have seen in meetings throughout my career. We’re all in this together, after all.

I’ve never seen software written with such efficiency.

07.12.12
by

Staying Current

During nearly ever job interview I’ve ever had, on the phone or face-to-face, I’ve been asked some form of the question, “How do you keep your experience current?”

Sometimes (emphasis on sometimes) this is asked by someone who seems impressed that I have such knowledge on a fair amount of “new stuff.” More often this is a genuine question (and a very good one) asked of an interviewee in an effort to gauge the type of software engineer that this candidate may be. Does this candidate have a desire to keep current with emerging trends? In many ways this is a unique necessity in the world of software engineering as a career.

Software engineering is a discipline that requires a real love of the work. Its not something that anyone hoping to find easy employment stability along with a solid paycheck should pursue (warning to those considering Computer Science).

So the question remains: How do you “keep current?”

No matter how much I love software, the fact remains that I have a life outside of work: family, friends, hobbies… Its not easy, but its necessary. One guy suggests learning a language every year. I like to pick up books that look interesting, read Stack Overflow and listen to podcasts. Woe to the “software engineer” who wishes to dismiss all emerging technologies as gimmicks or buzzwords… Such engineers will quickly find themselves (if lucky) performing maintenance work on antiquated legacy code. As long as software is something of a hobby, something that is personally rewarding, all of the above should be easy enough.

While all of the above is good, and its even better to have a pet project, I have found that nothing compares to working around extremely intelligent people for a company that is serious about software.

11.17.11
by

Appropriate Checkin Comments

I read this post today with a list of funny checkin comments today. Some of them are funny simply because of the lacking description. Here are some comments I’ve seen in my personal experience:

  • many small changes
  • Microsoft IE sucks!
  • cleanup
  • oops
  • fix the bug

Worse, I’ve seen entirely empty changeset comments.

The above lists, along with those found on the funny checkin comments page provide some examples of inappropriate commit comments. Why? They are unprofessional and lacking in detail and meaning. Some projects are audited and reviewed by external third parties. As a project manager or architect, would you be embarrassed for  an auditor to see the comment “fix sucky code?” I would. Even worse than being embarrassed, there is a productivity problem that can arise from poor checkin comments.

What is an appropriate comment? An appropriate comment must (minimally) have a few things:

1. Appropriate level of detail about the change, including why the change was made, what impact there may be, etc.
2. Appropriate to the changeset. Along with this, a single changeset should, as much as possible, reflect a single ticket or change. Many lazy developers check in a large set of code with a number of intertwined changes that are unrelated. When it comes time to revert a particular change or track a defect this creates problems, and ultimately it defeats one major purpose of version control.
3. Details about the completeness of the change. Generally a changeset should complete a ticket or work item, but this is not always the case. If there is remaining work to be done, “TODO” items or further functional requirements that impact the changeset, this should be noted.
4. Finally, perhaps the most important part, the checkin comment should refer to a ticket. Not all changesets have tickets written, sure, but in general, if the ticket is a defect, enhancement or requirement implementation, there should be one or more tickets that are related. Any modern version control and ticket system will be able to tie these together.

One developer writes:

Many developers are sloppy about commenting their changes, and some may feel that commit messages are not needed. Either they consider the changes trivial, or they argue that you can just inspect the revision history to see what was changed. However, the revision history only shows what was actually changed, not what the programmer intended to do, or why the change was made. This can be even more problematic when people don’t do fine-grained commits, but rather submit a week’s worth of changes to multiple modules in one large pile. With a fine-grained revision history, comments can be useful to distinguish trivial from non-trivial changes in the repository. In my opinion, if the changes you made are not important enough to comment on, they probably are not worth committing either.

Getting developers to write good checkin comments seems to be an ongoing battle. In the business of writing software, its easy to convince oneself that checkin comments are a waste of time. The real waste of time is later, when trying to track the introduction of a defect or trace requirement implementation to code. There is simply no good excuse for lacking checkin comments.

Is the checkin comment “cleanup” appropriate? Yes, in some cases, as long as its true. If I am cleaning up formatting of code, including things like indents and spacing and correcting whitespace, then yes, “cleanup” is an appropriate changeset comment. Generally, however, real comments are required.

[Vistamix: The Humor of Code Checkin Comments]
[Loop Label: Best Practices for Version Control]

11.07.11
by

What Every *GOOD* Developer Should Know: Quality Assurance

I’m currently reading “The Clean Coder,” and Robert Martin puts emphasis on the importance of software engineers incorporating QA practices into their regular work much better than I can. Here are a few quotes of his on the subject:

“Software is too complex to create without bugs. Unfortunately that doesn’t let you off the hook. The human body is too complex to understand in its entirety, but doctors still take an oath to do no harm. If they don’t take themselves off a hook like that, how can we?”

“Some folks use QA as the bug catchers. They send them code that they haven’t thoroughly checked. They depend on QA to find the bugs and report them back to the developers.  Indeed, some companies reward QA based on the number of bugs they find. The more bugs, the greater the reward.”

“…So automate your tests. Write unit tests that you can execute on a moment’s notice, and run those tests as often as you can.”

I am really enjoying this book. Its a fun and easy ready.

[Amazon: The Clean Coder]

11.06.11
by

What Every *Good* Developer Should Know

I came across this guy’s blog post titled “10 Things Every Good Web Developer Should Know.” The post is geared toward web developers, but it did get me thinking a bit about the more general questions. I’ve noticed shortcomings among developers (myself included) for a many years. What are some of the things that all GOOD developers SHOULD be expected to know? This list is hardly comprehensive, but I can think of a few things right away:

1. Linux/Unix

If you can’t do basic editing in vi you may find yourself in for a world of hurt at some point. I’ve known many programmers who attempt to write software in the safety of their IDE running on Windows only to find severe problems when it comes time to deploy on the server (and the server is typically some flavor of Linux). I recall a fellow software engineering student in my college days saying of his code, “It all works, it just won’t compile!” It sounds silly, right? Assuming software that is written, build and deployed in Windows will built and deploy just fine in another OS is equally as silly (yes, this goes for Java as well).

2. How to debug

Duh, right? Not so. I have helped many, many engineers with basic debugging. I don’t know if it is that I am particularly good at debugging (I’d like to think so), or that (some) others are particularly poor at it, but for most of my career I’ve heard, “Matt, this isn’t working, can you help?”

Generally this question is asked by an engineer who has spend a fair amount of time staring at the screen hoping to gain some diving inspiration and fix a bug. It never works this way. You have to be willing to dig into the code and actually find where the error is. Look through that stack trace! Run the debugger! When all else fails, start sticking print lines all over your code! Staring at the screen will rarely reveal a complex bug. A compile error, sure, but a bug, no.

3. Basic knowledge of C/C++ and or Assembly

In the day of virtual machines, powerful IDEs, scripted languages, OOAD and encapsulation on top of encapsulation on top of encapsulation, it can be too easy to write code and never understand exactly how much stuff has to happen for that code to work its magic. I have not written anything in assembly since college, and I have not written C code for 10 years, but I rely on my knowledge of the low-level “stuff” every time I write code. It helps to understand fundamentals of computer science, optimization, memory handling and what exactly makes all the magic of a 4GL come together. Many people get by without knowledge of assembly language, sure, but these people will not be “superstar” engineers… They’ll be programmers.

4. Version Control

There’s no excuse for not using version control. I would say it borders on negligent not to.

5. HTML, CSS, Javascript
This one may seem like another no-brainer, but I have run into many developers over the course of my career who simply do not have anything more than the most basic understanding of HTML.

6. System Administration

Just the other day sendmail quit working for me. I use sendmail to alert the team about project activity in Redmine, Subversion and Jenkins CI. I run project management software that is served using Ruby and Rails, Apache and Tomcat. I have written perl scripts for handling batch jobs and specialized email alerts. I have written bash scripts that tie in to various subversion triggers. I have installed Ant, Maven, Git, Subversion, Tomcat, Apache, GTK, GCC… You name it… All with NO help from a Linux administrator. Like it or not, these activities become the responsibility of the lead software engineer. If you embrace it and enjoy it, life will be easy. If you are lost, and waiting for the help of a system administrator, you may be in for a very long wait!

7. Database Design

EVERY good programmer MUST understand things likes normalization, joins, foreign keys, natural keys, sequences, race conditions, locking, and on and on. We cannot rely on a database designer. Even the largest companies I have worked for have, the ones with database administrators, have little if anything to say about database design. Database design is the responsibility of the software engineer. A poor design can cripple what may otherwise be good software.

8. Quality Assurance

Our goal as engineers it to deliver a high quality product with no defects. We all know that there will be defects, but this fact does not change the goal.

9. Communication, Documentation, Technical Writing

Even if your company does have the means to hire a dedicated technical writer, that employee will have no idea what your code is doing. Strong documentation is on the engineer (us). I never had to take a technical writing class in college. Fortunately, writing is something I enjoy. For the engineer who hopes to never have to write a document, he or she is likely to be very annoyed in this career.

10.14.11
by

Best Software Developers

From Kawseq Consulting:

An average developer can produce software 10 times as fast as the worst developers. The best software developers can produce software 10 times as fast as average developers. That means the best software developers can produce software as much as 100 times as fast as the worst developers.

1. Hiring cheaper developers actually costs much more up front than hiring the best developers, because you have to hire many more of the cheaper developers to get the same job done.

2. Hiring cheaper developers actually costs more later because you have to spend a lot more developer time fixing the higher number of bugs they put in.

3. Hiring cheaper developers means waiting longer to get working software because of the additional build-test-fix cycles to fix the larger number of bugs.

No matter how much time you give them or how many you throw at a project, cheaper sotware developers cannot produce code of the same quality as high quality software developers. You cannot expect a large number of cheaper software developers to produce a high quality result, just as you would not expect to hire 10 house painters and get them to produce the Mona Lisa. The lower quality produced by large teams of average or poor software developers inevitably leads to software that is more expensive to maintain and develop down the track.

That last bit hits the nail on the head!

Also, cheaper developers do not necessarily supplement more experienced and qualified developers. It just means that the better software developer has to rewrite the mistakes made by the less skilled team members.

Kawseq Consulting: Why Quality is More Important Than Price

09.18.11
by

Test-Parallel Development

Here’s a post (albeit dated) where a developer lists a few problems with test driven development. There are plenty more where that came from. What I’ve found works better is a hybrid approach, where we write tests at the same time as code (or just after). The idea behind pure TDD is one of those that sounds good on paper but is difficult to implement practically. Developing to the test means that we abandon some of the best parts of Agile by again tying our hands to strict requirements (this time the requirements are automated tests that don’t work until the code required is implemented). While I am a big supporter of functional automated tests and their inclusion in CI, I don’t think pure TDD is practical. A much better approach is to write functional code and tests together.

The biggest problem I have with TDD is included on the Wikipedia entry on the subject:

Test-driven development is difficult to use in situations where full functional tests are required to determine success or failure. Examples of these are user interfaces, programs that work with databases, and some that depend on specific network configurations. TDD encourages developers to put the minimum amount of code into such modules and to maximize the logic that is in testable library code, using fakes and mocks to represent the outside world.

Fakes and mocks are fine, but I prefer to spend more time implementing tests that run against real world conditions. Also, most all applications that I work on include a UI and/or database. Often, database and UI design occurs alongside all other development.

Taking HTMLUnit as an example, how often do we know what form and input names will appear on a page before we implement it? The same is generally true of database design. In an ideal world, pure TDD would be a great approach. In the real world, where I work, I need more flexibility. This being said, I think most software teams aren’t anywhere near this being a problem. Most have yet to spend appropriate time on automated tests.

09.02.11
by

Is the Software Medical Device World Ready for Agile?

To begin with, I don’t see any real reason why software medical device manufacturers should fear Agile. I do, however, see some stipulations that need to be made. Here is a rather dated article on the subject (from 2007) : Agile Development in an FDA Regulated Setting.

The author of the blog post concludes:

It seems to me that Agile methodologies have a long way to go before we see them commonly used in medical device software development. I’ve searched around and have found nothing to make me think that there is even a trend in this direction. Maybe it’s that Agile processes are just too new. They seem popular as a presentation topic (I’ve been to several), but I wonder how prevalent Agile is even in mainstream software development?

Since the article was written (4 years ago), Agile has clearly gained a solid foothold in mainstream software development. With companies bound by medical device FDA guidelines, however (or even IEEE, ISO 9001, ISO 13484), there may be some understandable fear on new approaches. What seems to happen is that the known process becomes the only trusted process, and adoption of anything knew leads to so many questions that it is simply pushed aside (regardless of the potential benefit to the company).

The “twelve principles” of the Agile Manifesto include:

  • Customer satisfaction by rapid delivery of useful software
  • Welcome changing requirements, even late in development
  • Working software is delivered frequently (weeks rather than months)
  • Working software is the principal measure of progress
  • Sustainable development, able to maintain a constant pace
  • Close, daily co-operation between business people and developers
  • Face-to-face conversation is the best form of communication (co-location)
  • Projects are built around motivated individuals, who should be trusted
  • Continuous attention to technical excellence and good design
  • Simplicity
  • Self-organizing teams
  • Regular adaptation to changing circumstances

Uh oh. A few of these principles are very likely to send upper management, at least those that are used to their traditional waterfall SOPs, running for the door. But who says we can’t make modifications where we need to?

I suspect that much of the resistance to Agile methodologies is closely tied to a fear of change. Upper management trusts that which they know, despite some of the obvious shortfalls.

08.10.11
by

Is there ever a reason NOT to use an Artificial Primary Key?

I found this post on the subject of choosing a primary key. While Java Persistence Annotations allow us to use any field we want as a primary key (as long as it is naturally unique), is there a good reason to use anything that is not a surrogate/artificial primary key?

There are plenty of fine examples for natural primary keys: SKUs, usernames, email addresses, and so on. While these may work fine as a primary key insofar as they satisfy uniqueness requirements, there are some drawbacks, the biggest being the fact that uniqueness may not be guaranteed.

This post lists the reasons against using natural primary keys with 10 very good points:

  • Con 1: Primary key size – Surrogate keys generally don’t have problems with index size since they’re usually a single column of type int. That’s about as small as it gets.
  • Con 2: Foreign key size – They don’t have foreign key or foreign index size problems either for the same reason as Con 1.
  • Con 3: Asthetics – Well, it’s an eye of the beholder type thing, but they certainly don’t involve writing as much code as with compound natural keys.
  • Con 4 & 5: Optionality & Applicability – Surrogate keys have no problems with people or things not wanting to or not being able to provide the data.
  • Con 6: Uniqueness – They are 100% guaranteed to be unique. That’s a relief.
  • Con 7: Privacy – They have no privacy concerns should an unscrupulous person obtain them.
  • Con 8: Accidental Denormalization – You can’t accidentally denormalize non-business data.
  • Con 9: Cascading Updates – Surrogate keys don’t change, so no worries about how to cascade them on update.
  • Con 10: Varchar join speed – They’re generally integers, so they’re generally as fast to join over as you can get.

So while on the surface it may seem simple to use a seemingly unique field for a primary key (a username on a domain, for example), it can be disastrous later on. Con 6 above is the big one, but Con 7 is something people don’t seem to think about as much. We can enforce uniqueness on any field we want, be it a key field or not… That said, I really cannot think of a good reason to use a natural key (other than developer laziness, which is in fact the key reason why bad code tends to be written in the first place).

[Stack Overflow: Deciding between an artificial primary key and a natural key for a Products table]
[Rapid Application Development: Surrogate vs Natural Primary Keys – Data Modeling Mistake 2 of 10]
[Wikipedia: Surrogate Key]
[Wikipedia: Natural Key]

08.06.11
by

Valuable Unit Tests in a Software Medical Device, Part 9

I thought I was done, but here is yet another good reason to incorporate complex function automated testing: Validation of multiple Java runtime environments. Fabrizio Giudici proposes this as a solution for testing with Java 7, but we can always take it a step further, verifying multiple OS environments as well. Of course, this requires that we have those build environments available (easy enough, using virtual machines).

06.26.11
by

Brain Rules: Why a Daily Standup Should Be in the Afternoon

I’m reading a book right now called Brain Rules. In it, the author discusses how after a number of hours sitting in front of a computer our brains literally start to call it quits for the day. This happens long before our typical 8 hour workday is up. If your job involves something that requires intense focus, such as writing software, you’re already well aware of this.

I wasn’t thinking of this when I planned my team’s daily standup meetings at 2 in the afternoon. Perhaps a bit selfishly, I was just thinking of when I like to have a break in my work. It turns out that I was probably motivated by the fact that this is a time of day when I feel the need for a change of pace.

I’ve found that when I am head-down into programming, the worst thing to deal with is an interruption. I’ve also found that I can only be hyper-focused on programming for 4-5 hours at a time (at best). After a while I just start to get tired, and its time for a break. The book Brain Rules offers some interesting insight into why this is.

Back to the stand up meeting: Long ago, when I first heard of daily “stand ups” I was alarmed. The last thing I needed was yet another meeting to interrupt an already busy day. What I didn’t understand was the fact that a daily stand up meeting achieves a few important things:

1. It actually reduces interruptions. People who may interrupt otherwise are encouraged to put the interruption off until the meeting.

2. It encourages work. I, and others, always feel like we want to have something to say at the stand up meeting.

3. It discourages long meetings. If the owner of the meeting (the team lead) is wise, he or she will insist that the meeting last no longer than 20 minutes.

4. It provides a much needed break in the afternoon, and an opportunity to refocus. Sometimes, if you live somewhere beautiful like North Carolina, its even a good idea to make the stand up meeting a walk outside. A little exercise and fresh air works wonders after sitting focused on a computer screen for hours.

[Amazon: Brain Rules]

06.24.11
by

Continuous Integration on Software Medical Device Projects, Part 9

Using a CI Environment to Replace the Traditional DHF

Naturally, an important part of continuous integration is having a CI build that can be checked regularly for continued build success. This is probably what is commonly though of as the key benefit, but there is much more to be gained. Any continuous integration environment that is worth using will allow the team to incorporate packaging of key project items with each build. This includes important documents, tests (both manual and automated test outcomes can be packaged), requirements, design specifications and build results (deployment packages, libraries, executables, installers, etc.). The important thing to note here is the fact that, used wisely, the CI environment can provide a snapshot of all project outputs at any given point in time. Hopefully it is becoming clear that this gives us the possibility of automated DHF creation. Not only that, we have a much more detailed DHF throughout the life of a project and not merely at a point in time in which a particular freeze was performed.

Tests

The continuous integration server should include unit tests (and by unit tests, I mean functional level automated tests) that provide a level of self-testing code such that any build that fails to pass these tests at build time is considered a failed build.

Packaging

 Continuous integration output need not (nor should it) package only built objects. We can leverage CI build integration with our version control system to package everything required per our design outputs (21 CFR Part 820.30 supbart C (d)), design review (21 CFR Part 820.30 supbart C (e)), design verification (21 CFR Part 820.30 supbart C (f)), design validation (21 CFR Part 820.30 supbart C (g)), design transfer ((21 CFR Part 820.30 supbart C (h)), design changes (21 CFR Part 820.30 supbart C (i)) and even our design history file (21 CFR Part 820.30 supbart C (j)).

Read it all:

[CI on Software Medical Devices, Part 1]
[CI on Software Medical Devices, Part 2]
[CI on Software Medical Devices, Part 3]
[CI on Software Medical Devices, Part 4]
[CI on Software Medical Devices, Part 5]
[CI on Software Medical Devices, Part 6]
[CI on Software Medical Devices, Part 7]
[CI on Software Medical Devices, Part 8]
[CI on Software Medical Devices, Part 9]

06.24.11
by

Continuous Integration on Software Medical Device Projects, Part 8

Build Script Creation

Ant should automatically determine which files will be affected by the next build. Programmers should not have to figure this out manually. While we will use an IDE for most development, we must not rely on the build scripts that are generated by the IDE. There are a few reasons for this:

  • IDE generated build scripts are not as flexible as we need them to be (it is difficult to add, remove and modify build targets).
  • IDE generated build scripts often contain properties that are specified to the environment in which they were generated. Along with this, something that builds okay in one work environment may not build when committed and used by the CI build or when pulled into another environment.
  • IDE generated build scripts very likely lack all the build targets necessary.
  • IDE generated build scripts may rely on the IDE being installed on the build machine. We cannot assume that this will be the case.

The Ant buildfile (build.xml) should define correct target dependencies so that programmers do not have to invoke targets in a particular order in order to get a good build.

Triggering Builds

As noted, we will use Jenkins-CI to automatically perform a CI build every hour if there is a change in the repository. The system will be configured to send emails to the development team if there is a problem with the build (i.e., if a changeset breaks the CI build). It is anticipated that the CI build will break from time-to-time, however, a broken build should not be left unattended. A broken CI build indicates a number of possible problems:

  • A changeset didn’t include a necessary library or path.
  • A changeset caused a problem with a previous changeset, and merge of the changes must be address.
  • A unit test failed.
  • The CI build server has a problem.
  • The build script failed to build the new changeset (missing library or required inclusion).

In my experience, the most common cause of a broken CI build is a lack of attention to the build script. Each developer is responsible to making certain that the ant build scripts are up to date with all required changes. We cannot rely on the build scripts that are generated by an IDE. There are certainly more possible causes that could be added to the above list. It is a good idea for each developer to trigger a CI build immediately following any Subversion commit to ensure that the CI build has not been broken. If a CI build continues to be broken without being addressed, the team leader and/or project manager may revert the offending changeset and re-open any related issue.

06.24.11
by

Continuous Integration on Software Medical Device Projects, Part 7

Build Labelling

A build is labeled with a predetermined version number (e.g., “2.0”) and with a Subversion changeset number. The beauty of this is that we have a build that is associated with a particular changeset and, by association, an entire set of project documents and sources (as long as we put everything in a single version control system). Once again it should be clear how beneficial such a setup is when thinking in terms of a DHF. No longer do we have to assign a particular team member to fumble through documentation, ensuring that the proper documents are copied to some folder. In fact, we have very little overhead; Our CI server did all the heavy lifting of us!

Mixed Revisions are BAD!

The changeset number, placed in a property file at build time (by the ant build task). If the changeset number has the letter M at the end (e.g., 3001M), the currently checked out fileset is a “mixed” revision.

The current working copy changeset number is easily viewable in TortoiseSVN or at the command line with the svnversion command. It is expected that during development and testing a mixed revision will be used at almost all times. However, any final build must not have the letter M in the build number.

If the letter M does appear in the build number of a formal build, it indicates that there are items in that build that are not up-to-date in the repository, and therefore the build cannot be duplicated using only a changeset or tag. To avoid this, the continuous integration server should be used to create formal builds, and it should be configured to use only a current changeset and no locally modified files.

Ant Build Targets

Build.xml is the Ant build file. It is located at the root level of the project source tree. The following build targets are necessary:

  • build – Compile all code, creating .class files
  • dist – Call the build target and package all code and necessary configuration files, creating a .war file (web application archive).
  • test – Execute automated tests
  • clean – Clean an existing build and dist (remove previously build items)
  • javadoc – Generate Javadoc
06.24.11
by

Continuous Integration on Software Medical Device Projects, Part 6

Build Scheduling
Jenkins-CI allows teams to set up a project so that a build is performed whenever a change is committed through a version control system. The “Poll Version Control System” option is selected to do this. From there, the team must set up a schedule so that Jenkins will know how often to poll the version control system. Jenkins can be scheduled to poll monthly, daily, hourly, or even every minute. However, I would recommend building hourly (if and only if there has been a change committed).
In the project view, there are lists of the build history. I’ve set up build scripts and artifact archiving so that you can get the build at any point. The Java archive files will often be set up to be self-contained. This just means that all the “stuff” that is needed to run the application (classes, libs, properties, etc) is archived in that single file, and you can always grab it from the CI environment.

To get a current jar (Java executable) go to the build history. A build with a red circle next to it failed. A build with a blue circle is a good build. Click on the most recent successful build and you will see the build #, date and build artifacts. You can download the jar files from here,
This is how we will handle projects in general when someone wants to get a build. There will be more details on tags and so on going forward. We can modify the CI environment to include any artifacts that we want, just as Javadoc or documentation (as long as it is something that we can pull out of Subversion).

Builds are created locally by developers for a number of reasons, however, such builds must always be considered informal. Any build released for formal testing, at the end of a Sprint or iteration or upon project completion must be done in the controlled build environment. Using self-documenting features of code (such as Javadoc), it can be wise to incorporate the output of this extra documentation into the DHF. Why not? There was little overheard in doing so and the benefits are substantial.