Example implementation of data pipeline based on serverless offerings by AWS. Data arrive to AWS Kinesis, processed by AWS Lambda and then stored in AWS DynamoDb. Check out “Simple serverless data pipeline using AWS Kinesis and AWS Lambda“.
Quite old news, but I added miniconda to chocolatey. If you need Python on Windows and do not want waste your time to setup wired stuff to compile packages like NumPy, you probably better to use Anaconda or Miniconda which is the same but with smaller number of packages pre installed. You however, always can install pre-built packages with ‘conda install’. So.. to install Miniconda for Python 2.x, use:
choco install miniconda
for Python 3.x
choco install miniconda3
Do not forget, after installation issue
conda update conda
Few weeks ago I completed nano project using NodeJS (express, mongodb, browserify, gulp).
May be half a year ago, Amazon eventually implemented something like Web and Worker Roles from Azure.
This year Build 2015 is exceptionally interesting.
Huge disappointment here, because of circumstances that is not under my control, I didn’t got v2. So only v1 real experience for the moment.
There were chances to get v2, so I downloaded SDK 2.0. Few interesting findings here:
I tried to use D3. Well… I am not sure that I like it’s API. Some strange intersections with JQuery and AngularJS. I was using D3 to visualize joint points in Kinect. Looks quite nice, but also feels a bit slow. Probably this is by design.
Example snipped that I came up to ensure that all additions/removes/updates are rendered,
I am on the final stage of my labs project for Lviv Polytechnic. I still hate API. But results looks much better.[youtube https://www.youtube.com/watch?v=Q46NjJcPZjk%5D
I am doing https://courses.edx.org/courses/DelftX/FP101x/3T2014/info from edX. It does not take too much time and looks quite promising.
Last few weeks at ELEKS I am playing with Kinect to help our UX team to implement exciting stuff they need. So my finding here:
On next week I will have Kinect 2.0 on my hands. I will probably have much more interesting things, but for the moment here is it:
It is first time I work for production AngualrJS. My findings could be naїve, but anyways.
Suppose you are in situation when you need to edit and then run some Python code, or C++ or whatever. And you need to this from your Windows Phone. What you will do?
Okay, most obvious solution is just to run one of the compliers that send your code somewhere and return result. You can find few of them in store. Most of them are quite low quality, but they work. Unfortunately, this options does not work for me. In my Python scripts I am using NumPy and none of the “compilers” have one preinstalled. So the question is custom environment.
Requirement of custom environment leads to custom VM somewhere™ in cloud. I already have had a few VMs on Azure, so I decided just to fire up another one. I created Ubuntu VM.
Through SSH I configured environment:
sudo apt-get update
sudo apt-get install git
sudo apt-get install python python-numpy python-scipy
Sidenote: this might be temporary issue, but. apt-get might fail with some wired 503 errors:
E: Failed to fetch http://azure.archive.ubuntu.com/ubuntu/pool/main/s/suitesparse/libamd2.3.1_4.2.1-3ubuntu1_amd64.deb 503 Service Temporarily Unavailable
E: Unable to fetch some archives, maybe run apt-get update or try with –fix-missing?
well… Azure distribution of Ubuntu, preconfigured with Azure own mirror for packages archive and looks like some packages are missing. To fix this replace all “azure.archive.ubuntu.com” to something like “archive.ubuntu.com” or “ua.archive.ubuntu.com” in “/etc/apt/sources.list”. One-liner do this from askubuntu:
sudo sed 's@http://azure\.archive\.ubuntu\.com/@http://archive.ubuntu.com/@' -i /etc/apt/sources.list
Now we have VM in somewhere™ in cloud. Next step is to connect from Windows Phone. Search gives few SSH clients. I tried most of them. They all actually looks the same(I think they use same terminal emulator) and only difference is keyboard. I selected Admin Shell and everything else is just self explanatory.
Vim works fine too.
Enjoy! Happy cheating on exams!
For me this is third attempt to complete ML course on Coursera , wish me lack so I have time to eventually complete it. In general I prefer to use Python for my studies, I like IPython, numpy and just Python, but for this course Andrew Ng decided to go with Octave, so I need Octave environment.
Octave does not have Windows distribution on home site. So you need to grab one from Octave Forge. Installation is easy and after all you will have octave.exe on your PATH (if not, just add bin folder to your PATH).
There are few items, that I found important for daily use.
– By default Octave starts in non interactive mode. You can type your commands and do math, but first error will terminate process, effectively. You need interactive mode, to do so just add -i when you start octave. In interactive mode you will have more chances to survive
– Octave outputs huge welcome screen, after you read it and can repeat it word by word at 3:00 AM, you can turn it off. Just add -q when you start octave. Personally I created cmd file that add both of this params for me.
Unfortunately, console Octave cannot plot graphs inline, this actually means that it shows plot in different window, with quite wired life time. Comparing to the other environments like IPython, wxMaxima or R Studio, I was sure that I will be able to find something like this for Octave. Actually I was wrong, there are nothing useful, but anyway, I spent some time to investigate and will put here aggregated information about Octave environments.
Octave Workshop is firs item that pops up when you search for Octave IDE. Project seems to be inactive, home site is down. If you will take a look at downloads section and find 5 downloads, it probably all y me.
In general it works. It has panel that shows variables(good), but actually without values(bad). It bundles quite old Octave 2.9.4. And does not plot inline.
Dead end. No value at all.
Commercial IDE for Octave, costs $49 for students.
Good things are: has variables view, can show actual values of variables. Uses Octave that installed on your system.
Bad things: on my system does not plot at all, when plot, still does not plot inline or at least to one of the tabs, it plots to separate window, that makes it quite unusable.
Well, my verdict. I would take one for free, $49 is over-over price(take R Studio as base of price judgment).
It is not in Internet any more :(.
Yeh, that is true. There is magic that enables Octave in IPython. It uses system installed Octave. It can output variables, it can work almost interactively. And it can plot inline! And of course it can exchange variables with main Python session, so after doing something in Octave you can easily play with results in vanilla IPython.
For me most visible problem with IPython as environment for Octave is requirement to put “%%octave” on every cell that actually octave cell, but I can live with that.
It is obvious, there is not single IDE for Octave that actually meets my requirements. Neither free, nor commercial. Until I will find something useful, I will continue to use console Octave in interactive mode and will also give a try for IPython Octave magic.
Disclaimer: I just completed Data Analysis course on Coursera, so this tricks could look very naїve.
Main reason is that RStudio makes it more visible of what are you doing now. Plot something and this plot will be right behind your eyes. Load data and you will see list of your variables. Invoke help and voilà.. help will not go away just because you typed few commands. And what was most important for me is data view, when you can see you data, you can feel it. You can also consider:
It is kind of unreadable when your model looks like:
lm(superData$AgeOfTheElephant ~ superData$SizeOfTheTail + superData$NumberOfLegs + superData$Location)
Better to use:
lm(AgeOfTheElephant ~ SizeOfTheTail + NumberOfLegs + Location)
Use detach to detach data. Attach works good, you can use it in any operations, but I do not like such side effect heavy tricks. My favorite is just to use:
lm(AgeOfTheElephant ~ SizeOfTheTail + NumberOfLegs + Location, data=superData)
In console just type “?lm” to get help for lm, if nothing appear try “??lm”, this is full text search for the help. If you load data packages, you can also use help. In most cases help provides valuable information on the structure of the data, its roots or even history.
You might know what Markdown is. If you don’t it is simple write and read friendly syntax for formatted text. RMD is Markdown with R extensions. Very useful to generate report of the analysis.
This is project scaffolding system for R. Looks very good, but as for me too havy on newbie stage. Also RStudio have it is own project system, but honestly I do not understand how this two things play together. I hope to play more with this stuff later.
Most important function.
ask questions there or on Stackoverflow. There are plenty information about R language on internet.
This is just suggestion, but use some style of the code. In R there are many naming styles, looks quite dirty. So you can follow Google’s R language code style guide as a base.
Also you might need R .gitignore file – here you are.
I own Surface RT. This post is all about my subjective opinions about it.
With just announced Surface 2, only #2 is fixed. Surface LTE will fix #3.