26 Nov 2009

Reply to SQL is dead, Long live SQL

I read the article SQL is dead, Long live SQL and I have to say I really liked it. It spoke about the NoSQL movement and how relational database appeal to analysts.

This struck a cord with me. I always believed that what the regular applications play a small part (perhaps a very small part) of what happens to the data through out its lifetime.
After the data is inputted for the first time, it is analyzed, dissected, made sense of and placed in several reports and KPIs that are called for several times a day/week/month.

The business that paid for the application to be developed needs that data. It is not trivial to them at all. In fact, they may possibly make long term strategic decisions based on that data (in an ideal world). Hopefully, they use that data to improve how they react with their customers and the outside world, making them a better and more profitable company.

Which in turn, makes them hire the same developers to write them some more applications. It’s all a big cycle and data helps keep that cycle moving (or at least, that’s how I see it).

So if you are a developer and you are reading this, remember that even though it pains you to write SQL code or use an ORM, it’s all part of an important cycle that makes the world a better place. That, or you might get paid more. Whichever is more appealing to you.

6 comments:

  1. clive boulton17 March 2011 at 07:30

    And this struck gold with me:

    "I always believed that what the regular applications play a small part (perhaps a very small part) of what happens to the data through out its lifetime."

    Business/ERP apps need SQL to do collections: Order Lines > Order totals. Yet, most access is read only for reporting.

    To massively cut down costs (10000x) consumer enterprise applications will need sharded NoSql databases, just like Facebook / Amazon to compete with Oracle / SAP.

    Salesforce uses a database on a database pattern to solve this. Assuming Occam's razor holds true, it should be possible to build the SQL collections between two or N NoSQL databases.

    My point is collections are only needed to create consistency for brief moments.

    ReplyDelete
  2. Jonathan Levin17 March 2011 at 12:17

    Currently there is no easy way (or no way at all) for business users to query data held in NoSQL type solutions. Please refer to the following link: http://howfuckedismydatabase.com/nosql/

    Plus, I don't really understand how you can compare what facebook and amazon do with the rest of the small, medium and even big businesses. It's like saying what Facebook/Amazon do is correct for my 5 person development shop.

    ReplyDelete
  3. clive boulton21 March 2011 at 15:52

    Couldn't Business users could query NoSQL KVS via API...(they do ok searching Twitter).

    SIRI used Darpa technology to develop relational inference queries. Similarity thinking there can be a way to query collections for business orders. (Apple has purchased SIRI) http://bit.ly/ffHl9h

    Small, medium especially could benefit from radical collaboration on Facebook scale to compete with big business. http://confusedofcalcutta.com/ Salesforce.com chief scientist is headed in that direction.

    ReplyDelete
  4. Jonathan Levin21 March 2011 at 16:52

    When I said business users, I meant more of the power business users that like things in excel, pivot tables and drill-down graphs.
    With NoSQL you need to decide ahead of time what the reports should be, ask developers to develop it and then present it to the business users when at that point all they can do is view it.
    This is ok for quite a few cases, but you do not have a situation where you can just sit down and analyse the data.

    I also disagree that if small business use this technology, they can compete with big companies. It would probably be so very difficult to implement that by they time they finish, they would be out of business (or a good chance that they will have neglected their major customer's feature requests).

    ReplyDelete
  5. Jonathan Levin21 March 2011 at 16:56

    Also, there are plenty of open source BI suites you can set up with none or very little money.

    If you're talking about money on hardware to power the DBs for these, you can also get those relatively inexpensively. You may need to buy ones with more 32Gb memory instead of the 8Gb you were hoping to buy as well as better hard disks. But I think in developer time, it most probably would be a fraction of the cost.

    ReplyDelete
  6. clive boulton21 March 2011 at 19:11

    Thanks for the insight into business users (I call then business developers or power users). They have no patience, don't care about elegant architectures, just want tools to solve business problems and get results.

    I'm trying to bridge convenient yet sophisticated applications for business using NoSQL data stores.

    Force.com internal design uses an Oracle database-on-a-database (2000). 10 yrs on should be able to use someones NoSQL at much less cost. http://bit.ly/ehd8Gy

    ReplyDelete