Spark 2.0 is here

After months of eagerly waiting, it’s finally here! I’ve been using nightly builds for the last few months, but for those not willing to live on the edge, Spark 2.0 is now officially out. I could highlight all the wonderful changes and enhancements, but the good folks over at Databricks have done such a good job, I’m not going to bother. Check out their post here. Suffice it to say that there are over 2,500 changes and performance has been drastically improved in almost all areas. There are new and simpler ways to do many things, but backwards compatibility is still very much a focus.

IFRS9 is coming! Are you stuck hoping your legacy vendor can deliver?

The banking industry is under immense pressure to modernize the way in which risk is calculated for financial accounting purposes. IFRS9 means big changes in both regulations, reporting requirements and ultimately portfolio strategies. With a January 1st, 2018 deadline looming, many bankers are understandably nervous about having the supporting systems in place to meet the deadline.

Unfortunately, many have pined their hopes on legacy vendors with long track records of cost over-runs, late deliveries, and failures. This is no accident, most legacy vendors are pitching ancient technology, in some cases over 40 years old,  to solve modern business problems. Not only is this incredibly inefficient, it leaves customers with brittle systems, with only those within reach of of their retirement knowing how to support them.

IBM’s Canadian Government payroll overhaul, a system name Phoenix,  is but just one recent example. IBM chose to use an ancient payroll system, built on top of an ancient database platform for the foundation of this massive government “Modernization” initiative. Now hundreds of thousands of federal government employees are facing a crisis, trying to get money they are owed. In some cases employees haven’t been paid in over 6  months, and are close to insolvency. Why are things such a mess? Development and maintenance on legacy systems is so complex and costly, it is nearly impossible to be agile when dealing with the complexities of modern business problems.

Solving requirements for IFRS9 using the state of the art technology, is a far better use of resources. Not only are you able to address your requirements much quicker, for a small fraction of the cost and complexity, you are also building towards the systems of tomorrow.  For example, instead of relying on a dwindling pool of SAS programmers to develop and maintain your IFRS systems, start fresh using the latest high productivity tools. Tools such as Cassandra, MongoDB, Python, and Spark and simple interface stacks like HTML5, AJAX, Anguar.js and Node.js, enable rapid development, testing and deployment of complex workflows. Resources that can develop systems using these technologies are far more available in the market, and in many cases less expensive.

The clock is ticking, hopefully everyone makes it across the finish line in time.