Key Takeaways from QCON London 2018 – Monday
This short article contains a few key take aways fro this years QCON Conference in London, obviously only covering the sessions that I attended.
Opening keynote monday morning was primarily on the importance and essence of Machine Learning.
Most important take away is that you really should not ignore ML even if you think it’s not important or interesting to you. Machine Learning will be a competitive factor
You should learn Python, Theory, Pytorch, Kaggle, Tensorflow. Take the Coursera course by Andrew Ng
Drivetribe: A Social Network on Streams
Besides awful echoing sounds, key takeaways are as follows:
- Log events using Kakfa -> https://kafka.apache.org
- Stream process using Apache Flink -> https://flink.apache.org
- Store data using ElasticSearch -> http://elasticsearch.org
- Serve api using Akka Http -> https://doc.akka.io/docs/akka-http/current/index.html
First 20 minutes was overview, followed by 15 minutes of Q&A, have no clue why they had this part early on not very well prepared.
Finally came to the real world example. Here one would expect a true demo, but instead it was powerpoint code, seriously… The example wasn’t even really showing anything of the architecture, it was merely a simplistic scala talk about making a bump functionality.
Nice to get the overview but it could have been done in 15 minutes or so, it was overall a waste of time.
RUST 2018 – An Epoch Release
Came to the session, knowing very little about RUST, but thought it could be an interesting intro for me.
- A language for systems programmers
- Things that you would use C++ for for example
- Blazing Fast, No segfaults, threadsafe
- Made as bit more high level version of system languages, hence approachable by for example java developers
Some of the talk was on build process, breaking changes and what comes in 2018.
So didn’t get so much about the actual language, but still interesting enough to dive a bit more into.
Also good speaker, so that alone made the session interesting
New features, areas :
- Webservices. RUST is used to program micro services quite a lot
- Async/Await as inspired by ES6
- WASM – WebAssembly . This is quite interesting, since web assembly could be a new way of making frontend solutions
- CLI command line applications
- Embedded Devices
Gonna look more into RUST i think, it’s quite interesting
Kubernetes – Crossing The Chasm
Having already deployed software on a kubernetes cluster, this session seemed to be an interesting one, where I can expand my knowledge on Kubernetes.
The session starts up pretty good, quick overview over what Kubernetes is, and something about adoption.
Quote “Cloud Native Is The Future”.
Moving on to some Real World Examples.
- First example was a startup that didn’t have legacy code and therefore naturally would fit in Cloud Native strategy.
- Second example is a company that had geographical barriers, needing to run the kubernetes cluster within the geographical region of china. The chinese firewall blocks google, so when kubernetes fires up it gets modules from google, which will not work. Ended up doing a more manual approach using Ansible. Kubernetes Federation came to the help. Federation has several kubernetes clusters and has a master cluster that can handle deployments on both sides
- Third was Ericsson, wanting to move to a cloud native architecture. Making all services to micro services , containerize and deploy in some sort of container. Ericsson chose to use HELM to make packages and hence be able to deploy their software packages.
- Forth was the horror story. A company with very high security restrictions. They were not allowed to adopt public cloud and has to be on internal closed net. The tools were not really build to work in an environment without access to the internet. When you create a pod, Kubernetes get’s a Pause container when deploying. So it accesses the internet, this doesn’t work if it has no connection to the net. Ended up recommending Swarm, Openshift or Similar
Some more challenges.
- Where is support for Microsoft ?
- Integration with vendor/legacy software ?
- Hybrid cloud. Some data on cloud, some data on premises.
Key takeaway is that Kubernetes is a “Whole” product with Mature tooling, Advanced Features and an Active Community.
Again a pretty good and skilled speaker.
Microservices lessons learned from a startup perspective
Session starts stating that there is always affecting circumstances.
Started with one team one tech stack and the system grew hard to manage. After this they changed to another strategy with smaller teams, smaller products, splitting monolith to micro services.
First approach was to make a co-existing service.
Lesson#1 . Don’t take too many steps at once. This will slow you down.
Lesson #2. Deferring solving Authorisation handling hurts. Make sure you have authorization services ready before all the other services. If you don’t you will start implementing authorisation everywhere. This is pretty much true in all kinds of system and is not really related to serverless.
Lesson #3. Less aligned strategy is expensive. separate apps, teams, services get’s bundled anyway. I’m not sure why this is a problem, isn’t it mostly a problem that it is deployed together. true micro services should be deployed separately.
Lesson #4. Data related overhead. Data have to be in sync. Solved by using Apache Kafka Streams.
Kafka is a strong streaming system and helps secure that all events that happens in the system, will be handled. A Scalable, Fault tolerant and fast solution.
Primary use of streaming platforms are
- Stream processing
- Data integration
Kafka gives a low barrier to entry for new service.
- No separate data storage to setup
- No extra copies to keep in sync
- No remote calls
- Materialized view always available
Started with a simple walkthrough of serverless and functions, for example : event driven, dynamic resources, billing per message, prototyping, focus on business logic.
Introduction to the providers : Amazon AWS Lambda, Google Functions, Azure Functions, Riff Serverless Platform, IBM Openwhisk, Oracle Fn, OpenFaas, Fission, Kubeless
On to coding demo, making a simple Spring boot application with a dependency to spring-boot-starter-function-web to show off a http function.
Basically you just implement Function<String, String> interface , this will make it a function that you can deploy to aws or riff as well as running locally as a spring boot application.
Next up is Project Reactor, the spring streaming framework.
Next up Riff is shown on Google cloud with 5 deployments in a kubernetes cluster.
Riff has a http gateway, that post messages to a kafka topic, the function controller then listens on this topic.