Awhile ago I was approached by Typesafe to do a developer Q&A session about Empathica's transition from .NET to Scala. I thought it would be a good opportunity to wrap up some loose ends from my Technology Change: .NET to Scala post of last year. The answers I authored were a little too verbose for Typesafe, so I decided to host it here because I think it compliments last year's post fairly well. Here it is.
Can you describe the event that led to you moving from .NET to Scala?
Empathica was mainly a .NET shop when I joined the organization, but Simon Palmer, our CTO had a passion for open source languages and frameworks. When the development team started a new project he encouraged us to try new things and consider different technologies. However, since almost all the development team had a predominantly .NET background, it was easy to justify its continued use based on the perceived learning curve of picking up something new.
In recent years, Empathica had done a great job at differentiating themselves from its competitors, but our sales team was still finding it difficult to sell our story to clients because our program lacked options for what we considered more traditional Customer Experience Management (CEM) features. In the summer of 2012 the executive team introduced a comprehensive plan to build products that would satisfy feature gaps in our program.
This was a major disruption to the development team, because we shifted our focus from extending our existing product suite (written in .NET) to starting several wholly new projects at once. It was this disruption which fostered several technology discussions across the team. I was on a team with 2 other developers. We were asked to create a proof of concept for a text analytics web application. We had planned on accessing this text analytics data from a different platform being written by one of our other teams, which we could access with an API. This gave us an enormous flexibility on technologies to write the web front end in because we really only needed a way to transform datasets and serve up a single page application in the browser.
During my teams technology discussion I encouraged my colleagues to propose something that was not .NET and was Open Source. We were confident that just about any technology we chose would be acceptable and that this was the perfect opportunity to try something new. We considered Ruby on the Rails framework, a simple node.js backend, and Scala along with the Play Framework. We read tutorials and gauged the development world’s opinion of these languages and frameworks. It was by no means an exhaustive analysis, but it didn’t need to be. We eventually reached consensus on Scala and decided to use it and the recent release of Play 2.0.
What was the resistance to the change? What were people worried about?
There wasn’t any initial resistance to the change because it was only one lone project and we chose the technology as a team. The project definitely got the management teams’ attention. Once we could prove to them that we were able to integrate with legacy systems and deploy into production they were onboard. Our CTO in particular was quite excited that we had introduced an open source platform to the development culture at Empathica. He became one of our strongest advocates. With his support it was an easy sell to continue using Scala.
The real resistance didn’t begin until we started using Scala and Thrift/Finagle for our back office API’s. As we wanted to put more and more business logic and data concerns behind an API we started encroaching on territory that was typically handled by another development team that was still exclusively using .NET. We had to pick one technology for the API and Scala had the support of the management team as well as a lot of momentum behind it from my team’s previous successful projects. Using Scala for the API was a major source of resistance with some of the other developers in the organization. They felt like they didn’t have much say in the decision. Debates started happening that echoed a lot of what was already being said by those who critisize Scala in the development community.
I’ll admit, I was a little worried too about continuing to use Scala since there were a lot of big differences from the Microsoft .NET stack. Not so much in terms of Scala itself, but in the tooling (or lack there of it). Being a .NET developer generally means you’re working with an all Microsoft development stack, with tools that work perfectly together. You’re connecting to other Microsoft technologies and things just generally work without much hacking about. In the summer/fall of 2012 the state of tooling (Eclipse IDE, SBT, tooling for continuous delivery, etc) presented a fairly steep learning curve for developers coming from the .NET platform.
How were those concerns resolved?
Tooling has gotten a lot better over time, but it’s still nothing like what’s available in .NET. I began advocating that people stop relying so heavily on IDE sugar and consider more traditional software debugging techniques. This includes things like
- Writing tests first so you can easily debug isolated parts of the system.
- Using a powerful logging framework.
- Deciding on a lightweight text editor that you find most efficient.
- Using a linux platform for access to powerful shells & tools to augment your toolchain.
This is how much of the open source community works and there’s no reason why we smart people couldn’t adapt our habits to suit the development environment we were in.
Our CTO heard both sides of the debate, and eventually made an executive decision that we were going to use Scala. As I said before, he was happy with the fact that we picked something open source, but more important than that, he felt we made the right technical decision about using Scala for an API and trusted us to make it.
It wasn’t all sunshine and rainbows. Some developers were just not happy with the situation and did not want to be involved in a non-Microsoft shop. It was sad to see people leave (on their own accords), but I think those that remained showed their commitment to the craft by their willingness to try new things.
We also changed our hiring process to include questions about how comfortable candidates would be in using Scala, or any other technology they don’t have experience in. Candidates that revealed no desire to try different technologies were filtered out. We even began hiring Scala developers specifically.
How long did it take people on the team to pick Scala up?
Few developers could deny that Scala itself was a great language. Coming from another great language, C#, there were many comparisons that could be drawn. Since Scala doesn’t really get in the way of how you write software (imperatively vs functionally) it was easy to hit the ground running and write working software. In that sense it didn’t take long for us to “pick Scala up”.
In terms of actually writing idiomatic Scala, well, that took quite some time develop. Most of us did not have too much experience writing functional code. Many of us had never heard of things like pattern matching and case classes. We considered the idea of bringing in inhouse training through the consulting services of Typesafe and BoldRadius (formally known as Tindr), but we thought we may get more long term value in enrolling in Martin Odersky’s Functional Programming Principles in Scala course on Coursera.
Our management team permitted us to spend 1 day a week during the span of the 7 week course to spend time watching the lectures, discussing material, and working on the assignments. It was optional, and obviously we still had to keep the lights on at Empathica, but the participation was fairly good. Some of us that were most enthusiastic about the course chose to enrol in the Reactive Programming course offered in the fall of 2013.
What did you find was the hardest thing to get your head around going from .NET to Scala?
Scala is chock-full of new and exotic programming concepts that took a lot of getting used to coming from a .NET background. Some examples include the myriad of ways to use implicits. It was a bit of a mind bender for me to grasp the different scenarios. I wasn’t too keen with them at first. I thought they could easily result in unmaintainable code! But as I was exposed to more scala libraries I started to see good implementation examples (i.e. the implicit thread local session in slick, or the implicit request in Play! actions).
Partial functions, pattern matching, and case classes took awhile to get the hang of, but again through patient reading and a lot of learning by example and reading other people’s implementations I was eventually able to appreciate these things.
I wouldn’t say there was 1 hardest thing to wrap my head around when transitioning to Scala. There were so many concepts that were foreign to me at the start that it was easy to feel overwhelmed, but that didn’t stop me from carrying on with my naive implementations. In fact it allowed me to iteratively write more idiomatic Scala solutions while still remaining productive on the job. Regular code churn is of course a sign of a healthy codebase and as we learned more about Scala there were plenty of opportunities to refactor solutions.
Scala is a rich language. In the community this richness is often confused with complexity. This is a consistent criticism of the language, but I find it largely unwarranted. There was an infamous blog post entitled “Scala - 1 star - would not program again” that fostered a lot of debate among Scala folks as well as the larger software development community. Martin Odersky’s reaction on the Scala internals group really resonated with me and is a view I share.
Scala's generality makes it both a threat because it *can* do tasks that are traditionally the domain of these other communities and an easy target because every community finds something that is foreign from their point of view. Java people will bitch about how academic FP is, JS and Ruby people complain about the types. Haskell people complain about the objects and that the types are not strong enough. And even though those communities would normally never mix, they find common ground in the opinion that Scala is a horrible language.
Clearly Martin was a little bit rattled by some of the debate going on at the time, but I share his opinion of how a lot of programmers are an idealistic bunch. Developers will strongly cling to concepts of one language over another because it’s so ingrained into their day-to-day routine. Just because new concepts challenge your way of solving programming problems shouldn’t have you running for the hills. Who’s to say there’s only one right way to do something? (except for the Dutch) In my opinion Scala has done an excellent job at marrying concepts from seemingly disparate programming paradigms. Sure there’s a learning curve to write so called idiomatic Scala, but I’ve found that learning to have made me a much better programmer and it has opened my eyes to many new ways to design software.
How do you think knowing .NET well helped your understanding of Scala? Were there any concepts in particular that you found to be similar?
I think that C# programmers will immediately find solace in the rich type system of Scala. C# being a strongly typed language itself, this is a fairly easy transition. Scala has done a fairly good job at making types as unobtrusive as possible.
In the C# 2.0 (released 2005) and 3.0 (released 2007) specifications a ton of great features were added to the language. Anonymous methods, lambda’s, type inference features, and LINQ were some of the most popular. The reception by developers was overwhelmingly positive. Many of these concepts have 1 to 1 mappings to Scala syntax. Scala’s collections methods offer all the capabilities of LINQ and more. Type inference is well done results in much less verbose code. Lambda’s have an even syntactically sweeter implementation than C#’s.
Yes, there are many core similarities between C# and Scala that make it easy for .NET developers to get their foot in the door.
What Scala frameworks did you end up migrating to? Why did you choose them?
We immediately started using the Play! web framework. At the time we adopted Scala, Typesafe had recently released version 2.0.4 and after a few days of experimentation we quickly built an appreciation for it. Being an MVC framework, it was in fact very similar from an architectural PoV to ASP.NET MVC. Since we collectively had a lot of experience with ASP.NET MVC it was easy decision to make for our team.
We spent a lot of time working with Thrift and Twitter’s Finagle library when we took on the task of rewriting our back office API layer. We wanted an easy way to interoperate between different tiers of our infrastructure and to communicate between different products in our software suite (largely written in C# on .NET). Thrift was a fantastic technology-agnostic choice for writing an API and Finagle provided a ton of great functionality to enrich our Thrift clients and servers. Finagle is written in Scala and has a very clean and easy-to-use API.
What were the biggest impacts on the team and their output after making the change?
Surprisingly there was very little impact to our productivity. Our team is comprised of some of the most intelligent, thoughtful, and creative people that I have ever worked with. All our developers work very closely with one another. We pair frequently and cross pollinate ideas all the time. Once we decided on Scala for our first web project there really weren’t any major problems we couldn’t handle. The project was a fairly astounding success both from a technical and business perspective.
When the rest of the development team started using Scala for new projects there was some of the resistance I mentioned earlier, but once we started slinging code the same thing happened and we hit the ground running.
I won’t lie, there were a few people who very strongly opposed the direction the development team was going with the Scala technology choice. As I mentioned before, we lost a few people who weren’t comfortable leaving the .NET world. It was unfortunate to see them go, but I don’t think it really had much of an impact on the productivity of the team as a whole.
Did the move give you a different perspective on the problems you were solving?
Certainly. We questioned a lot software design decisions after the move. I think one of the changes of perspectives was now that we have two different technologies how do we effectively share domain/business concepts between them without lots of reimplementation. It forced us to draw a clear line in the sand between product and backend infrastructure. This caused us to reevaluate how many of our apps were written (in .NET, Scala, and Java) and the net result was that they all became far simpler because they deferred much of their data access needs to our new API layer.
Strictly from a code perspective, we found that when writing Scala we had much more emphasis on employing FP solutions to problems. Especially after we encouraged people to do the Functional Programming Principles in Scala and Reactive Programming Principles courses. For me it was a fairly radical change of thought and I would still say I’m not the most effective practitioner of FP. That said, I still strive to use FP where appropriate. I found that when we put in the extra effort to solve problems this way we ended up with more robust, less buggy, and easier to understand solutions.
What business impact did the change have for the company?
Let’s face it. Running a .NET shop is not cheap, especially when you’re also invested in other Microsoft infrastructure such as SQL Server and developer tooling. Using Scala immediately opened the door to a whole wide world of OSS and rich Java ecosystem of libraries, servers, and applications. Empathica hosted much of its own infrastructure and as such had a fairly hefty licensing bill to Microsoft. Obviously you can’t change your whole infrastructure overnight and expect to remain in business, but we chipped away at systems and started recouping significant costs.
Empathica had a pretty good buzz with the technical community in Toronto in a large part due to our CTO Simon Palmer’s vision, but once it was known that we were venturing away from Microsoft and into new and exciting like technologies like Scala it gave us an even better reputation. We started interviewing candidates from all kinds of different development backgrounds, not just those with .NET. We changed our line of questioning in interviews to gauge how open people were to trying new technologies. We hosted a Scala meetup with the help of local Toronto Scala shops, BoldRadius, and Typesafe.
“Disruption” was one of our core tenants at Empathica and although this tenant was likely attributed mostly to our disruption of the CEM marketplace, it also was an invaluable quality possessed by many of our business and technical teams. The Scala technology choice was a disruptive event. Early on it was a scary path to be on, but quickly we realized that the innovation it bred was going to pay dividends in respects to our product suite as well as the development of our technical team.