Image via Wikipedia
History
Since its inception, the company has been a traditional LAMP stack. Over time, some internal business applications were needed to be developed. The initial platform to solve this requirement was Microsoft Office and specifically Access database and some Excel sheets. Later on, more and more VBA programming was added to the Access database. At some stage, a decision was made to move to VB.NET and port the VBA/VB6 code to it.
So far, this story is not that unique and I am sure many of you reading this can either relate to it or know of examples from other places.
There were issues moving to VB.NET from the start. It took over a year to finally move to it with many complications and a lot of resources invested in it during that time. After some time, the resources depleted and 1 developer was left to maintain the system.
With time, the application stagnated and the 1 developer could not keep up with the bugs and requirements for new features.
The Present
At this stage I came into the company and I decided that some serious work had to be done to improve the internal business application. I had made a plan to fix and improve the existing code base but saw that it would take a considerable amount of time and energy as well as expertise that we did not have.
At this point, I took a look at other parts of the company. The company has a very good history with open source products. The relatively new PHP team was accomplishing quite a lot of things. All the servers are on various flavours of Linux and maintained well.
The only parts that were holding the company back were closed source technologies. Specifically, incompatibilities between 3rd party applications, printers and forced networking standards that come with closed source. The cost for .NET developers was also an issue.
It seemed to me that the culture of the company is more aligned with open-source and ultimately, that would be the direction the company should follow. The only problem was that, because of initial convenient and easy-to-use application, the company was locked in to a now costly and constricted system.
The system that was chosen was PHP. From one point of view, PHP was the obvious choice, considering the expertise the company has. From another point of view, selecting a web-based technology to replace fat clients with rich GUI, isn’t the most obvious choice.
Considering all the different points (and after being grilled by the executives a bit), the decision was made mainly because of our strong history, achievements and expertise with PHP and Linux. The difficulties that PHP might have to replace the VB.NET system were researched and solutions were offered.
The Official Reasons to Move to PHP from VB.NET
- To cut down costs and unify technical support expertise. It is difficult and costly to support multiple systems technologies.
- To reduce complexity and speed up development time. The company has had a lot of success delivering products and features with PHP and believes that this success can be replicated for the internal business application.
Full Steam Ahead
When making this system it is very important to make sure that the system will be very stable, maintainable and flexible for future changes. It would need to prove that it can overcome the difficulties that the old system had troubles facing. After that, development speed is crucial as all new features will have to be put on hold until all the system (or at least some parts of it) is rewritten.
So the important factors were the quality and flexibility of the code as well as speed of development. I decided that in order to achieve these goals, we would need to use frameworks, tools, add-ons and coding standards. The idea is to leverage existing open-source products and best practices to save time and ensure quality. It is also important to make sure that by using the same tools and following the same coding standards, the code will be maintainable for many years to come.
The List of Technologies
I have compiled a detailed list of technologies, open-source products, frameworks, tools and coding standards we plan to use to get this project done properly. All of these are supposed to lay the foundation for a rigid and structured development process.
- LAMP Stack - PHP 5, MySQL 5.1 – for improved replication and partitioning, Ubuntu Server and Apache Web Server
- Zend framework – This is something the company has chosen a long time ago and it’s something we plan to continue with.
- APC – PHP accelerating to speed up bootstrapping.
- Memcached – To speed up database interactions. We came to the conclusion that memcached sped our whole system by a factor of 40.
- Doctrine ORM – I didn’t want us to spend a second of a developers time on mapping database tables to classes, so I wanted an ORM. Doctrine seemed to have a better website then Propel, so we chose that. So it’s literally a case of judging a book by its cover.
- YahooUI Javascript Framework – After my developers being biased to jQuery and I was biased for ExtJS, we decided to choose YahooUI as it was somewhat close to the rich GUI that our VB.NET uses now. We chose this framework to try to keep javascript development to as low as possible by using already made widgets and libraries from the framework.
- PHPUnit – This is one of the important parts of the foundation we need to start building the new system. If we cannot have some guarantee that the system is stable then what is the point of the whole thing.
- phpUnderControl with CruiseControl (Continuous integration server) – “Continuous integration describes a set of software engineering practices that speed up the delivery of software by decreasing integration times”. phpUnderControl uses PHPUnit, PHPDocumentor, PMD, PHP_CodeSniffer and CheckStyle.
- Subversion SCM – We obviously, need source control management and we are already familiar with subversion. We had considered Git, but didn’t choose it in the end.
- Zend Coding Style – The development team decided that in order to organize and standardize their code, they want to use the Zend Coding style. I also understood that it will later help with phpUnderControl with its style checking.
- Scrum with ScrumWorks – We use Scrum and we find it very effective. The application we use for it is ScrumWorks Basic. Even though we don’t think it’s the best one, it works for most cases and is quite convenient.
- Selenium with PHPUnit extension – for front-end testing. Selenium can test front end pages/forms and javascript functionality. Selenium can be called from PHPUnit which will fit with the automated testing suite.
- Acceptance Testing for our User Stories – This is a standard we liked. We like to specify with the user stories the acceptance test or success criteria for this user story to be considered done.
- Fat-Models, Skinny Controllers – A standard we are all in agreement is a very good practice.
- PHPUnit TestDox for documentation - I wanted the tests to be the actual documentation. All the requirements and behaviors of the system should be the actual tests of the system. PHPUnit has a way to extract the test name and generate documentation for them. We will also use PHPDocumentor and Dokuwiki for anything else.
- SOAP/REST– We will use SOAP/REST for any component that might need to be called from another location. With all the talk about SOA, I decided to just implement the minimum use of it, which is to use it when other locations need to use this functionality. This is instead of having the code in 2 places and both try to get data from the same database as the point of integration.
- FireFox/Prism - We can dictate which default browser we plan to support. Prism is also quite a cool application to make our application look like… a real application.
- Validations for HTML and CSS – Even though we don’t need to make our internal business application SEO friendly, it still might be a good idea to run some validation on it.
- LDAP (linux) - We though it might be a good idea to have this for user logging and authentication, but the brightest idea would be that we would then know which printer is closest to that user.
- Nagios for Web Page monitoring – We can monitor if the website is up and get advanced warning if something is wrong.
- Log4php – a ready made logging library ported from java.
Conclusion
We are all quite pleased about the decision and look forward to start working on this project. I hope to learn as much as I can from this experience and I am sure the development team feels the same way.
I strongly believe that PHP with the addition of other open source products can deliver the same if not better results then VB.NET/ASP.NET would and I plan to put my money where my mouth is.
For writing this article, I did some research into Enterprise PHP, but did not find too many (current) articles about them. I hope to start seeing a bigger trend towards Enterprise PHP applications
I will keep bloging about the project when interesting things come up and I keep the right to update the list I made in case we decide that some parts of it are not as good as we thought
I agree with all the points you made. The only thing that I would rethink is the Javascript framework. I know Yahoo shows a lot of widgets out of the box on their website, but it just seems cluttered and slow (however the latest version shows huge improvements). I am not saying you should choose the small footprint libraries you already mentioned, but maybe there is a middle choice - maybe Dojo :)
ReplyDeleteI don't think ORM is a good choice. Do you really think that ORM solves your problems? PHP supports multiple string literals. So SQL is natural choice (compared to Java).
ReplyDeleteJava sucks when coding SQL because it does not support multiple line string literals. Lot of string concatenation need to be done when working with a simple SQL. Therefore Java developers need to move SQL to XML configuration like iBatis or use ORM to reduce cost of maintenance or sometimes to avoid the so-called object-relational mismatch
That does not happen in PHP. SQL is extremely powerful right?
I'm also developing an application meant to replace currently used spreadsheets and MS Access database. Many choices I've made are similar to yours. I did stick with ExtJS. What bothers me, is it gets pretty slow with large datasets to be displayed in grids...
ReplyDeleteMany thanks for showing me Prism! Look like it will be very useful!
I think the list of tools you have given is exactly what I would have chosen in your position. Well I would probably have chosen a more complete MVC framework (i have not really used ZF for that) like Symfony, CakePHP or CodeIgniter. Well actually at my company we mostly use OKAPI (okapi.liip.ch), which is ultra lightweight. Guess it comes down if you prefer to strip down a full featured framework or add to a lightweight one.
ReplyDeleteAs one of the developers involved in this project, I have to address some points:
ReplyDelete- Javascript framework - Dojo would be a poor choice for us, because the documentation is incredibly bad. jQuery was one of the alternatives that we considered, as we already use it in other projects, but YUI has more widgets out of the box. We may take another path as the project proceeds, depending on real-world performance, but that's something we can't plan for.
- ORM - the primary reason here is to avoid writing SQL. If we don't have to write SQL directly, there's less chance of running into situations where we haven't escaped parameters properly, etc etc. Plus it allows us to hit the database in an object-oriented way, which writing raw SQL wouldn't.
- Framework - well, we've chosen Zend as again, it's already used in other projects. It's not my personal choice (I've previously described it as PHP trying to be Java), but as we have prior experience it seemed to be the logical choice. Hopefully we can get past the major pain points quickly.
It's a very exciting project to be involved in and hopefully we'll be able to quickly get stuck right in.
Hi,
ReplyDelete1) Why not use Zend_Log instead of Log4PHP? At the very least make sure you write a wrapper around it, so it's simple to switch at a later date (e.g. MyLog::debug(....)).
2) I keep meaning to write an article comparing Propel and Doctrine. Doctrine certainly seems to have more buzz around it - and the Propel requirement to go through a code generation hoop is abit of a pain (although I'm curious as to whether this means Doctrine will have worse performance).
3) Is it possible to render HTML 'pages' in the VB.NET application - allowing you to migrate features (rather than an all in one big bang?) - I'd expect there might be UI issues, but it might make a transition easier?
4) LDAP seems a good choice - I'm certainly looking at using it internally for us too (too many passwords for too many things...)
5) My (limited experience) would make me err on the side of SOAP over REST - as I think there would be less 'leg work' involved to create the interface to start with (SOAP can be created through normal php doc comments, REST probably requires you write XML 'views').
Finally, with regards to the recent PHPWM meeting - we all know that for Business web application development in PHP Pale Purple should be your first choice ;) Those who try to game Google for personal reasons should be punished! :)
Hi David
ReplyDelete1) I thought log4php had more logging functionality. It had daily logging for example. Although I only mentioned it because I was familiar with the java version.
3) Its not feasible for us at this stage to do that. Lack of time, resources and knowledge.
6) You are right, gaming Google should be a punishable offense, however, you can keep your link on my blog, its fine ;)
Hello Jonathan,
ReplyDeleteFound your blog on Planet MySql. I work in a weird world where I use Microsoft for everything EXCEPT database where I use MySql.
It sounds like your article is as much a reason to start with a more "industrial design" as it as much a problem with Microsoft's lower end tools.
Creating apps with Access, Excel, VB6, just really don't scale. VB.net is okay if you use Option Strict and Explicit, but my guess is that it was not used. And I'll admit that I am blessed to never a Windows Forms app and deal with the horror that ensues. I do think that if you start with C#, Asp.Net (spice with jQuery), MySql, NUnit, an ORM such as NHibernate or even Entity Framework, Subversion, and other tools and you can get a lot more done than Starting with Access and Excel. I have this book on “ASP with VB 6” that almost every chapter began with an introduction and a following comment that said “… and this is hard.”
I think the big take aways are:
1) Go with what you know - if you know LAMP and related tools you would waste a lot of time moving to MS, or vice versa. Jumping platforms is a big time investment.
2) Research the tools before you start. It looks like you have a good knowledge of what tools work for you and that worth more than what most people realize.
3) Bad, overly complicated software projects are hard to rework regardless of platform. Both Architecture Astronauts and Newbie’s are guilty. I think we all know of and have made samples. :)
That said fair MS has been lagging WAY BEHIND in understanding what makes Open Source attractive. It’s hard to match the speed of delivery and affordability that many of the Open Source projects have. They have made some steps. They have a long way to go. Don’t know if they’ll get there.
Need to check out YUI, and I subscribed.
nice.
Delete@David Goodwin
ReplyDeleteI know that .Net 2.0 has a browser control available but it's not straight forward to implement. You can also host .Net windows forms controls in IE, but your stuck with IE.
I know that I am starting to look into Adobe Air for an impending project I want to build with Flex. I think they just released a Linux verson. Air supports Flash, Html via Web Kit, and PDF.
Nice article. PHP has evolved into a very solid tool for building robust web applications, and surely enough is ready to go mainstream Enterprise (not that it hasn't already; look at Facebook, Yahoo, and many others who are successfully building web apps using PHP).
ReplyDeleteTo the list you've added I would add CakePHP, it is our primary MVC framework that we use in our company.
PHP developers should write more on the topic of enterprise applications, where there is a lean towards .NET and Java based frameworks . Someone should explain those .NET/Java lovers (not that I don't find them powerful, I've coded with both) that it is not the only way to develop web applications. Open Source stack that we have today can be as much powerful, if not even more powerful.
Thank you again Jonathan, for bringing up the issue.
Hey Jonathan, it's 3 years now and after some time the project probably came to an end - what was a good choice and what turned out to be bad?
ReplyDeleteDevelopment working fast and every thing OK so far? Would you change some of your decisions in a retrospective?
Thanks for making it clear! :)
Many thanks for showing me Prism! Look like it will be very useful! I am a php web application developer and always searching for the stuff like this, so as to increse my knowledge everytime.
ReplyDelete