- 616.538.8747
In software development—and large software projects in particular—there’s a pretty big problem. They fail too often. Far too many come in late, over-budget, and lacking any sort of maintainability.
2010 Michigan Agile and Beyond was held last Saturday, and it was all about one of the most popular ways companies are dealing with this issue: Agile Development. I was glad to be able to attend, and very impressed with the presenters and their ability to explain what Agile is really all about.
There’s a lot to be said about Agile— whole books are written on the subject—but I’ll attempt to quickly explain what it does and where it’s an appropriate philosophy. These are the ideas that stuck with me and the ones that mainly revolve around development, though there’s much more to the entire Agile method, especially regarding communication. The main points of the “Agile Manifesto” can be found here: http://agilemanifesto.org/
A commonly cited source of problems is simply the changing world and that requirements change with it. At the beginning of a project, developers ask for a full specification of what the customer wants. The customer may even provide a spec that’s pretty complete. But as anyone who has worked long as a developer will know, it is inevitable that these requirements and specifications will change and be misinterpreted. These issues frequently aren’t discovered until the very end of the project when things are supposed to be finished. Customers aren’t really the ones to blame for this. The world simply changes, and large software projects are too complicated and have too many exceptions and rules for any one person (regardless of talent) to be able to visualize them completely, even in a specification document. It’s simply too big.
This problem often reveals itself in the code. If the developers are using a “waterfall” method, where the customer often doesn’t see the product at all until the end, the developers will often have to “hack in” a bunch of changes at the end to fit what the customers want and still stay under deadline. This leads to a messy codebase.
On the other hand, “rapid prototyping”—giving customers a new version of the software to look at very often during the project—often isn’t sufficient to solve these issues either. Those prototypes are usually developed with the mindset of “I’m going to refactor this later and make it more maintainable.” Unfortunately, the refactoring and simplifying rarely actually happens. When it does, it often introduces new bugs as dependencies are forgotten, another symptom of the system being too large for one person to visualize.
Although rapid prototyping isn’t ideal, it can often work better since it at least acknowledges that requirements will change. It gives customers the ability to easily figure out what they want since they will see a working version of the software in front of them. However, as stated previously, on its own it doesn’t always lead to a quality product. To solve this issue, Agile introduces Test-Driven Development to the rapid prototyping process.
Test-Driven Development (TDD for short) is about automating the testing process. In normal development, testing is often confined to just a week or two weeks at the end of a project and is done entirely manually. Not only is this often cut or reduced as projects get closer to the deadline, but it’s a great deal of wasted man hours and must be repeated when changes are made to the system.
The idea behind TDD is that developers write tests for their code BEFORE writing the code itself. Initially, it may seem absurd that one would code something that would fail a test they had written just a few minutes previously. But the benefit of the test is not to ensure the code does what it’s supposed to right then and there; it’s mainly to ensure that the code does what it’s supposed to regardless of what else changes in the future. Large software projects are very interconnected, and changing one small part often has implications for many other parts. The tests ensure that when you change something, you aren’t accidentally breaking the other pieces of your system.
A philosophy of Agile development is that software projects should respond to change, and TDD is a requirement for this to happen. In non-TDD development, a customer will ask for a change to the software or add a new requirement, and the developer (if he’s careful) will have to check through everything that could potentially be affected to ensure that new bugs aren’t introduced. With TDD, he would simply make the change, run the automated tests, and get an immediate report of whether his change broke anything else. He would also be much more confident in immediately showing his change to the customer because he would know that he had not introduced any new bugs.
TDD also allows for greater ability to continuously refactor and simplify the code, which Agile encourages. Refactoring introduces the same issues as making small changes to the code: you don’t know what else in the system might be broken because of changes you make. TDD solves this in precisely the same fashion, because it can automatically test your code once you’ve made changes.
The advantages outlined here come down to this: much less manual testing, the ability to have very maintainable code, and the ability for customers to give constant feedback on the latest version of the software.
It doesn’t come free, of course. Agile is really an investment in coding infrastructure. There’s a good deal of initial work required to ensure the ability to release software to your customers on a regular basis. There’s also an ongoing cost of writing tests for every new piece of code. But this investment really pays off for many large projects that need to respond to change, because you end up finishing more quickly or further under budget due to the reduction in bugs, testing time, and misunderstandings. That’s not to say every project may finish more quickly though. For instance, very small projects may not get their investment back, because the projects might have simple requirements that don’t change.
Finishing more quickly is great, but even if that doesn’t happen on some projects, there are other advantages to this method. Let’s say, for example, that there are two identical projects—one attempted with a waterfall methodology, one attempted with an Agile methodology, and both doomed to go exactly the same amount over budget and past deadline. The advantage for the Agile project is that all parties would have known this outcome was coming much earlier in the process, and so would have been able to plan for it by reallocating resources or canceling the project.
Agile’s goal is to mitigate the risk that’s taken on by every party involved in software projects, and for many projects it’s very effective at accomplishing this. The statistics on software projects in general aren’t pretty, and there’s a lot to be said on the low bar some projects set for “success.” Hopefully, with further adoption of techniques like Agile, software projects can come to be seen as much less of a risky investment.
What really makes Elexicon valuable to clients is the ability to deliver a lot for a little. When it comes to web programming and development, this can be difficult, because there are so many ways a development project can go wrong. I won’t reveal all our secrets, but here are some of the important lessons I’ve learned that help me ensure Elexicon or our clients don’t waste time or money.
I enjoy developing elegant, clean, programming solutions, but sometimes cutting corners and doing something quick and dirty is fine. No one but me cares about how code looks - they care about how it works. On the other hand, if it’s going to cost the client more for me to maintain my messy code in the future, then it may be better to spend more time to provide a more elegant solution.
The lesson here: Context is everything, and programming in a bubble should be avoided, because the programmer often will know best the questions to ask that could save the client money.
As an example, say a client named Jane Dough needs to import some data into her big important database. She’s considering having Elexicon build her an automated process to do so. At this point, we would want more information. Some questions we would ask would be:
From the answers to these questions, we could help her determine whether the cost of doing the manual entry would be less than or greater than the cost of building an automated process to do so. Often, there are diminishing returns, where it may cost a great deal to fully automate a system, but to compromise and just partially automate a system is a lot cheaper.
Other examples of common questions I ask myself:
This applies to every level of the development process. It may seem obvious, but I still see it violated often.
On a higher level, don’t even think about coding your own shopping cart solution, blog software, or simple content management system. While it might guarantee your job while you have to maintain the behemoth of code you created, it betrays a lack of knowledge of the world around you. Instead of giving clients a system that’s built exactly to spec for a lot of money, you can often buy them a system that goes above and beyond what they want for even less money. It may take time to find the exact solution that fits, but that pales in comparison to the time you would spend implementing and maintaining your own solution.
There are definitely exceptions to this, but during the proposal process the key is to ask yourself: Does the client want something that’s similar to sites all over the internet? If so, someone has probably developed something that will save you time and the client money.
On a lower level, there are plenty of development tasks that can be automated or made easier by libraries. However, I would issue a warning: these concepts should be fully understood before attempting to automate them. I would not use an Object-Relational Mapper (such as Subsonic) if I couldn’t figure out how to code one myself, or use jQuery if I didn’t understand what makes it so powerful. It’s a delicate balance between libraries that save you time on trivial tasks, and those that will take too long to learn and/or restrict your ability to maintain and simplify your code. This is why it’s important to understand the concepts in the first place.
Thankfully, at Elexicon we have a team of project managers who are used to working with both design and development, and who ensure that our lines of communication remain open. Nothing has the potential to save yourself and clients more time, money, and headaches like proper communication, mostly because improper communication can be disastrous. This isn’t just about asking the right questions to be aware of the context. It also comes down to the ability to translate tech-speak into language that the client can understand. Not only will this ensure that clients understand what is being delivered, but it will make your process more transparent and trustworthy. It’s good to be able to answer the client when they ask questions like “Why does this simple development task take 20 hours?” in terms that anyone can understand.
Sometimes, you come upon tools and tricks that are so useful, you wonder how you survived without them in the first place. Here are some of mine.
My first tip may seem really trivial - but with all the CSS builds I’ve done, nothing has improved the way I do CSS more than keeping my styles all on one line. Basically this means writing this:
.style1 { margin-top:6px; font-size:22px}
instead of this:
.style1
{
margin-top:6px;
font-size:22px;
}
This makes a huge improvement in organization and ease of editing. It’s much, much easier to group related CSS items. Since the more spread-out version often separates styles with an additional break between them, your eye simply does not recognize the grouping as a distinct block of styles that belong together. Additionally, whatever comments you have to indicate groups of CSS styles are often off the screen. In contrast, look at a style block like this:
/* My group of CSS styles */
.style1 { margin-top:6px; font-size:22px}
.style2 { margin-top:0px; font-size:12px}
.style3 { margin-top:525px; font-size:42px}
.style4 { font-size:42px;}
.style5 { padding-top:42px;}
/* end my group of CSS styles */
In the more spread-out version, this group would probably take up a good portion of your screen - especially if the styles have quite a few rules. Instead, your eye instantly recognizes that the styles belong together and it takes up a whole 7 lines. You no longer have to spend hours scrolling around your 15 page CSS file (2-3 pages is usually what I end up with).
As a web developer, there’s almost no excuse for not having Firebug. It is the most useful front-end development tool that I’m aware of. Among other things, Firebug lets you debug Javascript, change CSS/HTML on the fly, and see all the requests you’re making while browsing.
Want to write extremely easy, concise, beautiful javascript? JQuery is the way to go. Sounds simple, but It’s basically a library that allows you to easily manipulate your HTML and CSS. It includes features like animation, ajax, and a huge library of plugins.
I’m not sure where I heard this saying originally, but it rings true in the context of programming and web development. It’s simple, but willingness to Google things can show you who the real problem solvers are. Google is possibly the best problem solving tool ever invented and it takes 10 seconds to consult it. If you have a problem with any aspect of web development, it’s almost guaranteed that others have had the same problem, so why reinvent the wheel?
Smashing Magazine is one great resource, and there are others like it if you look. They often have lists of great development tools and new trends in web development. More than many other programming jobs, web development rewards people who stay current on the latest trends and technologies.
Much has been made lately of the explosion in applications for the iPhone and its market-leading app store. We’ve just recently been trying our hands at it. I’m a developer, and I’m on the bandwagon. This is why.
Why is a simple, fairly low-powered platform a good thing? While it reduces the number of things that you can do, the restrictions of the platform also level the playing field. The internet has become so massive that it’s often difficult to start a new idea and compete without a large amount of resources. Mobile apps, with their simple platform, effectively put a cap on how effective the big guys are able to be. Indeed, there are many very creative independent developers taking advantage of the platform and producing some very successful applications.
This can be a double-edged sword. For instance, “Simple UI” doesn’t necessarily mean “Easy to design UI”. Apps with clunky or unintuitive UI are going to be left in the dust. UI is an area where the iPhone and Apple have really driven user expectations up. We’ve seen this recently with the unit conversion app that hit the Top 25 in the iPhone App Store, and I’ll delve deeper into that later.
This is the most obvious piece to this. Commercials for the iPhone do a very good job of reeling customers in because they really demonstrate that “cool factor.” I’m a developer, but even I’m often stunned into thinking “how did they do that?”. The recently featured “bump” app had this effect on me as a friend and I downloaded and tested it to determine how exactly it functioned. In this case, the verdict was that it wasn’t quite as magical as I thought (you can “bump” from across a room if you just do it at the same time).
Apps like this demonstrate that there’s still plenty of potential for exciting innovation. Unlike internet applications where you just have the keyboard and mouse, on the iPhone you get multitouch, a compass, a GPS, bluetooth, an accelerometer, along with the entire internet and all the amazing APIs available. Honestly, I’m still trying to come to grips with all the possible inputs. But the bottom line is that both developing for and using the platform is fun.
There’s plenty of cool things on the horizon as well. See: Augmented Reality.
The large amount of Flickr, Facebook, and Twitter apps out there demonstrate that APIs aren’t going away. But extending your site isn’t something only those big sites can do - it’s definitely possible to extend your own website with an iPhone app. If your website is interactive in any way, you can port some of that functionality to an app, similar to how the Facebook iPhone app works.
Look at your customers - is there a possible mobile application that could be of utility or interest to them? If so, developing it and branding it for yourself might be a good way to market yourself in an area where it’s possible few of your competitors have ventured.
Monetization is one of the biggest problems facing people with websites and web applications today who want to actually make money off of them in a non e-commerce fashion. Rumors persist of Facebook, Twitter, or Hulu turning to some sort of paid subscription model to become profitable. Money from advertisers may not be enough to satisfy infrastructure costs as web applications grow larger.
I’m reminded of one of my favorite new TV Shows, Shark Tank (fun, although somewhat cheesy). An oft-repeated maxim on the show is that all entrepreneurs must ask the question: “How am I going to make money off of my idea?”.
App stores provide a very attractive, easy alternative to the traditional ad-based model. Many apps don’t even provide that much more functionality than a simple browser does. But because the web browser on the cell phones will always be somewhat limited, many apps have evolved as a way to provide a much better UI. For instance, the iPhone Facebook app is simply a more attractive and user-friendly way to interact with their site than trying to navigate it with the iPhone Safari browser.
It’s obvious that there would be a huge uproar if Facebook began charging for its services. However, it wouldn’t be so out of the question for them to have released their Facebook iPhone app at $.99 (they didn’t, however). At this point, people still expect to pay for apps. The centralized payment structure makes it even more easy - not only for the customer but for the smaller entrepreneur or developer.
I’m certainly not saying that every app on the iPhone or other mobile platforms will make bushels of money, but it provides a very attractive alternative to those who are pessimistic about dealing with advertising as their main method of making money for their web application.
It’s also important to note that while the iPhone is attractive in it’s simplicity, ultimately the amount of money you can make off of a single idea is probably limited - currently, it seems unlikely that one could make millions off of a single idea. That’s a restriction that doesn’t exist as much on traditional web apps on the PC. This article has some additional perspectives.
This one speaks mostly for itself, and was a no-brainer for app stores to implement. Delivering upgrades and patches is always a headache for developers and the centralized store certainly simplifies the process.
I’m unsure how long this platform will last in its current form. As time goes on, most of the former points will be affected. More apps will get released, competition will increase and we might see a recession of paid apps in lieu of more advertising driven ones. The platform will certainly get more powerful and perhaps less accessible to the little guy as expectations rise. People eventually will get used to the hardware innovations and the cool factor, and perhaps they will effectively run out of hardware improvements after a while (how many more things can they pack into the iPhone?).
One thing to note about the possibility of increased competition - just because an app (or multiple apps) has been released doesn’t mean you can’t innovate and improve on the same ideas. Just recently a unit conversion app was released on the iPhone that made the top 25 apps (and $40,000 in sales in it’s first 2 weeks). There are countless unit conversion apps already available on the iPhone - it seems impossible that one was able to rise above the rest. Yet it happened, and it showcases how good UI design matters. There are many parallels here to the successful apple products - the iPod certainly wasn’t the first MP3 player but it was able to rise above the rest due to it’s very well designed UI.
For right now there are plenty of opportunities, especially for the small entrepreneur. We’re excited. If you or your company have a good idea, or want to put a spin on an old one, get in touch with us and we’ll see if we can make it a reality.
Elexicon is an interactive design and development agency based in Grand Rapids, Michigan...More>
