Digital-Thinking
A Programmers Blog about Machine Learning, Computer Vision and more
The Nanodegree program from Udacity offers some tech certificates, especially for programmers. One of them i called “Machine Learning Engineer” and with the github Student Developer Pack one month is free. So i decided to enroll this course in my semester break and…
Spark 2.0 data frames offer a very powerful way of accessing structured data in a SQL like way. But sometimes its hard to find the right build-in expression. So I would like to show some things, which is was dealing…
Spark 2.0 is now released. Time to move forward. After spending some time migrating the titanic project to the new version, it seems that Spark 2.0 does not change too much. Like said in the Spark Summit in San Francisco 2016…
When running Spark 1.6 on yarn clusters, i ran into problems, when yarn preempted spark containers and then the spark job failed. This happens only sometimes, when yarn used a fair scheduler and other queues with a higher priority submitted…
After getting good results with the Random Forest algorithm in the last post, we will take a look at feed forward networks, which are artificial neural networks. Artificial neural networks consist of many artificial neurons, which are based on the…
In the previous post i showed how to use the Support Vector Machine in Spark and apply the PCA to the features. In this post i wills show how to use Decisions Trees on the titanic data and why its better…
In my previous post i showed how to increase the parallelism of spark processing by increasing the number of executors on the cluster. In this post i will try to show how to distribute the data in a way, that the cluster…
Very interesting talk about Spark Performance:
This ist the third part of the Kaggle´s Machine Learning by Disaster challenge where i show, how you can use Apache Spark for model based prediction (supervised learning). This post is about support vector machines. The Support Vector Machine (SVM) is…
The naive bayes classification is a probabilistic classifier, which is used to classify a feature vector to a class. It does a probably wrong assumption, that features are statistically independent to each other. Anyway Apache Spark has implemented Naive Bayes and i…