Software Development Philosophy and Practice
There is no perfect development style, but over many projects I have developed a set of best practices and a philosophy that have served me well. I have worked on small teams or by myself on my projects. On the team projects, I have usually been the designer and project manager in adition to my other roles.
There are a lot of articles and books on application development style. This one is personal. It is about a philosophy and approaches that have worked for me over almost twenty years of delivering increasingly complex database driven applications. A lot of it will look like Agile, even though I have been developing this way since before Agile became a buzzword. However, I like the 2001 Agile Manifesto enough to include it at the end of this article.
I currently work in ASP.NET and SQL Server, so my examples will be based on Microsoft tools and servers, but good projects come from good planning, good project management, and good people, not some magic development platform.
This document is a work in progress. As I think about this subject, I will continue to fill it out.
My (bad) attitude on code-first development
I have had more than one developer tell me that the application should be developed first, and then the database should be generated to match. Microsoft even enables this foolishness. My rational for considering code-first foolish follows:
- If you have enough information to build a user interface, you have enough information to build the basic database.
- If you do not have enough information to build a user interface and a database, by definition you do not know what you are doing, so don't do it!
- Databases built "code-first" do not contain all of the data protection and audit features real world databases need, until someone expends a lot of extra effort. In many cases, the additional effort will be greater than if the database was designed at the beginning of the project.
- At this point some Agile folks may insist that they don't need no stinkin requirements, but they are wrong. Agile doesn't do away with requirements, it manages changing requirements by making change management central to the development effort. You still need to have enough original requirements to create something to change.
- You do not need to build the entire database or user interface at once. Pick a functional portion (a module) and concentrate on it. That is a central tenet of Agile.
If you don't know how to design a database, ask for help. Designing a database is a separate discipline, and if you need help, get it. Set-based programming in SQL to create views, stored procedures, and triggers is different from writing procedural code, so ask for help if you need it. There is no shame in not knowing everything, although some work environments make it seem shameful to ask for help.
I sometimes hear the excuse that the database is constantly changing. If you have a proper data layer, there is often only one place where data access needs to be updated. If the changes are extensive, the business and application code is going to have to change, and putting off all of the data access code updates until the end of the project is a recipe for failure.
If the database is changing so much that the developers really can't keep up, there is a major project management problem. This is a good time to consider the project manager. Either the project manager doesn't have enough authority or enough competence.
Justifying the project
Justifying the project begins with defining how the result of the project will make someone's job easier, save the organization money, make the organization money, or provide better data to the planners in the organization.
Next, the cost of the project needs to be balanced against the expected gain. That is usually called Return on Investment (ROI). The bigger the expected ROI the easier it is to sell the project.
Obviously this can require a lot of research, but if the project proceeds, the research will be useful. If the project doesn't proceed, the research will have prevented a problem project.
Any project, even a small one, needs to justify its existence. On a large project, deciding whether to continue is an ongoing part of the project. Demonstrate progress at every opportunity.
The basic business requirements should have already been part of the research required to justify the project. Now it is time to fill out the details.
Keep in mind that most projects evolve during development, sometimes because the business is changing, sometimes because the understanding of the requirements improves, and sometimes because of technical opportunities or problems. Modern development styles have largely been optimized to deal with this challenge.
The first thing to understand is that the requirements are constantly changing on most projects. Usually the changes are not major, but the documentation must keep up with the changes, and the effected stake holders need to be kept current.
- Requirements documentation should be online and current.
- Stake holders should be notified when a requirement affecting them changes.
- Requirements documentation needs to have formal version control, just like the code.
- There are many types of stake holders, so there needs to be a variety of documentation types.
Be sure to involve all of the stake holders
- Top management who have to support the project and need to be able to justify it.
- If it is a public facing Web site, do testing with people who represent average users to see what they need. Remember that you may be trying to reach more than one type of user.
- Administrative people in the organization who will use the system to manage the data and create reports. This could include top management.
- All of the managers, designers, and developers who will be involved in implementing the project. It is easy to forget that these people are stake holders, too.
- Other users within the organization, or people who depend on the application users.
Setting technical and design goals and standards
Specify an operating platform
The hardware and operating system are often a given based on the existing systems in the organization. Changing platforms requires new hardware, new software licenses, new competencies, etc., so it doesn't happen often.
For larger projects, especially within medium and large organizations, there are two popular platforms for internal applications and Web/database development:
- Oracle Java and Oracle database server software
- Microsoft ASP.NET and SQL Server database server software
There are several other popular platforms. The most commonly used are:
- MySQL or MariaDB, or PostgreSQL database server software (all are free and open source, although MySQL is owned by Oracle Corporation.)
- Apache or Nginx Web Server (free and open source)
- PHP, Perl, Ruby, or Python (free and open source)
- Very large systems have been developed on these platforms, but they are more commonly used for smaller projects.
- If you already know them, Java and ASP.NET are also viable for small projects, and have some advantages, especially in the programming tools.
Setting user interface design goals and standards
By the time the coding begins, the look and feel should be decided and approved. There should also be flow diagrams plus wire frames and/or prototypes so the coders know what to build and the end users know what they are going to get. All of this will probably evolve and change during development, so build change management into the project management. Agile has a lot of good ideas, such as having a user representative as part of the team, and dividing the project into small easy to change portions as separate tasks.
Setting coding standards
Coding standards are important, and there are a lot of them. Good standards should include:
- Document the code.
- Use a bug tracking application to keep track of bug fixes and code changes.
- Put comments in the code. Explain what the code is to accomplish, not how it does it.
- Code should be as self-documenting as possible. I ask myself "Could my students figure this out?" You may need to develop your own metric.
- Separation of concerns, a common scenario:
- Data access code in a data layer
- Business rules in a business rules layer
- Presentation in a presentation layer
- Layout in a layout layer
Data flow should be "clean". A data request should be routed directly to the business layer which passes the request to the data access layer, which returns a value(s) to the business layer which manipulates the data and returns it to the presentation layer. All requests for a specific data item should go to the same business code object. There should never be more than one way to return the same data, and all data access should be through the business layer.
The actual number and names of layers can be different in different platforms.
For example, the repository and unit of work patterns can facilitate automated unit testing or test-driven development (TDD) by adding another layer(s). I have used this on a couple of projects, and didn't find it to be much help. Your mileage may vary, especially if you work with it longer than I did.
The Microsoft development environment
Specify a development platform
From this point I am going to talk mostly about Microsoft tools, because I am most familiar with them.
Before getting into Microsoft land, the best free tools for Java and PHP development are based on Eclipse, an IBM sponsored free and open source Integrated Development Environment. Wikipedia has some articles that will give you a starting place. I do my PHP development in Adobe Dreamweaver, but it is not free.
In the Microsoft development environment the tools and servers are tightly integrated. That does not mean that you don't have choices of servers and languages, but it is easiest to use Microsoft tools to develop against IIS Web server and SQL Server databases.
Microsoft has a free version of SQL Server which includes Management Studio (database development). There is also a $50.00 Developer version of SQL Server which is functionally the same as the Enterprise Edition. A free version of Visual Studio (application development) is also available. Use the new Community Edition of Visual Studio which is full featured.
Visual Studio: Application front-end (executed in the browser) development
Visual Studio: Application back-end (executed at the server) code development
- C# or VB.NET (other languages are available in Visual Studio, including PHP)
- T4 for automating code generation
- SQL or Entity Framework for communicating with the database server
SQL Server Management Studio (SSMS): SQL Server database development
- Database design and development
SQL Server Integration Services (SSIS) using the Business Intelligence Development Studio (BIDS)
- Develop data transformation packages within a drag-and-drop interface. Customization through programming is also supported.
- BIDS is actually Visual Studio.
- SSIS is not available with the free version of SQL Server, but a $50.00 developer's version of SQL Server includes it.
SQL Server Reporting Services (SSRS): Creating reports within SQL Server
- I do have projects with SSRS reports, and some teams use them a lot. I usually do my reports as part of the application, but there are advantages to doing the reports in SSRS. You might want to check it out.
Building the Software Development Team
I have created a list of the required jobs to be done on a software projects. This list is fairly long since there are so many technical and interpersonal aspects to any development project. Sometimes all of the jobs are done by one person, sometimes each job requires a team. In any case, every one of these tasks needs to be completed.
Planning for changing requirements
Most of the time the client doesn't have a clear idea of what is needed, and even the best Requirements Analyst can't get every detail right the first time. Things start becoming clear when there is something concrete to show the client. When the client is provided with a prototype or an early version of a section of an application, the real requirements start to jell. Making this work requires:
- Constant effective communications between the client, the requirements analyst, and the developers.
- Good documentation. Good documentation is complete, to the point, and up to date.
- Responsiveness on the part of the developers.
Agile Software Development
Agile is a way of thinking about software development, not a fixed set of techniques or tools. Agile refers to being able to change quickly when it advances the project. There are techniques and tools that help keep a project flexible and help keep communications between the development team and the other stake holders effective, but don't be afraid to change tools, or even procedures, if you are not getting the results you want.
A few of my favorite books on the practice of development and development teams:
The Mythical Man Month, Frederick P. Brooks Jr., 1982
Algorithms + Data Structures = Programs, Niklaus Wirth, 1976
Manifesto for Agile Software Development
We are uncovering better ways of developing software by doing it and helping others do it.
Through this work we have come to value:
- Individuals and interactions over processes and tools
- Working software over comprehensive documentation
- Customer collaboration over contract negotiation
- Responding to change over following a plan
That is, while there is value in the items on the right, we value the items on the left more.
Kent Beck, Mike Beedle, Arie van Bennekum, Alistair Cockburn, Ward Cunningham, Martin Fowler, James Grenning, Jim Highsmith, Andrew Hunt, Ron Jeffries, Jon Kern, Brian Marick, Robert C. Martin, Steve Mellor, Ken Schwaber, Jeff Sutherland, Dave Thomas
2001, the above authors this declaration may be freely copied in any form, but only in its entirety through this notice.
My (Conrad Muller's) work on this page is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 2.5 License.