Latest Event Updates

Serverless: the new trend

Posted on Updated on

Recently the new hot buzzword seems to be “serverless”. There is a meetup group called “Serverless NYC”, and there have been a few “Serverless” conferences. Lambda is the new tool from Amazon that gets thrown around by developers at every meetup. But what are we talking about exactly? Just to be clear, even though the name is serverless, when you eventually get to the bottom of the tootsie roll pop, there is a server there. It’s just about who controls it.

The people who are using the services provided are still running their work on someone’s server.”With a serverless architecture, you focus purely on the individual functions in your application code. Services take care of all the physical hardware, virtual machine operating system, and web server software management.”[1] This allows the developer to focus on being a developer, and not  necessarily require the extra skillset of knowing, Linux, networking, and infrastructure. Since the focus here are functions, it should probably be some smaller project, or something that can be independent of the scope of a larger project.

The benefits of a serverless architecture is no need to deploy or administer any servers, install software, or run updates. That is handled by your provider. If you need more compute you can just purchase more consumption rather than build and deploy new servers, even if just virtual machines. You don’t need to architect for availability and fault tolerance  since that too is built into the architecture you are buying. Availability would refer to your service being up, while Fault Tolerance would refer to withstanding the failure of a sever.

With every upside there is a downside. Some of the downsides of serverless computing are:  1) multitenancy problems, which means that if the provider over provisions their resources and available compute is limited, it will affect your services. 2)vendor lock-in: if you design your services around the capabilities and api’s of provider A, then it makes it much harder to move to provider B. 3) security concerns – you don’t really know the status of the security of your host. You trust your provider, but for those with the need for absolute control or some regulations, they need to make sure the solution is viable for their needs.

While there are some downsides, there are many potential upsides. Serverless is here to stay and should be weighted as a viable solution for any deployment.

Passenger: the web application server for rails

Posted on Updated on

When I first looked up Passenger, a Gumdam-like robot icon appears. From the site itself, “Battle-hardened by some of the most demanding web apps for over a decade, Passenger is considered one of the most performant, secure and mature app servers currently available.” (1)

OK so what is it? Passenger is an open source web application server which handles HTTP requests, manages processes and resources, and enables administration, monitoring and problem diagnosis. But then question is, if I type “Rails S” why would I need passenger? Passenger doesn’t replace “rails server” in the sense of it being a test platform only. What “rails server” does is it uses an application server – such as Passenger – directly to server ruby files.

The default web application server activated when you use “rails server” is Puma. Looking at the Gemfile, you can see “gem ‘puma’, ‘~> 3.7’”. I think this is a great thing to realize, a lot of new Rails developers need to not only know about how to write code in Ruby and Rails, but also about the ecosystem that runs a web application, instead if using Heroku as a crutch. Standing up a server on a standalone box can be a great learning experience. With Linux you would use Nginx/Apache to host pages, as they are web servers. They provide the HTTP transaction handling and serve static files. However, they are not Ruby application servers and don’t  run the Ruby applications by themselves directly. Nginx and Apache are used with an application server, such as Puma or Passenger. What application servers do is make it possible for Ruby apps to speak HTTP. Ruby apps (and frameworks like Rails) can’t do that by themselves. So we need the application servers so the ruby apps can talk to http. But application servers aren’t as good as Nginx and Apache at handling HTTP requests because Nginx and Apache are better at handling I/O security, HTTP concurrency management, connection timeouts, etc. This is why a good Rails Engineer will still need to know about Linux, Apache, and Nginx.

Why choose Passenger over Puma? Passenger’s strength is that is also can host Python, Node.js, and Meteor apps as well. Passenger is the best choice when you have apps developed in several languages that Passenger supports and you want to consolidate your app server stack. Puma’s strength is that it’s easy to configure and mostly “just works” out-of-the-box. I think that after getting familiar with what Puma does, its better to look at Passenger for growth cases

Kubernetes, an introduction

Posted on Updated on

“Kubernetes is Google’s open source system for managing Linux containers across private, public and hybrid cloud environments.”(1)Now why is this important? A development deployment system will be encompassed of many different functions, not just a webpage, but databases, apis, worker resources, and more. The most important thing is that as long as you are using kubernetes to deploy, you’re not tied into a particular vendor. If you are having problems with say, digital ocean, you can move to Amazon. Or lets say you are on Amazon, and Microsoft Azure is giving you some extreme discounts that make migrating a financial win, then migration can be easy.
“Kubernetes, at its basic level, is a system for running and coordinating containerized applications across a cluster of machines.”(2) It is an orchestration workflow that helps define how your applications should run and the ways they should be able to interact with other applications or the outside world. Create, dev, test, and production environments, scale up for the holiday season, scale down when not needed, etc. You can even use layers in layers, making it a kubernetes russian doll, brining together both physical and virtual machines to make a cluster.
In an overview, the Master server: “acts as a gateway and brain for the cluster by exposing an API for users and clients, health checking other servers, deciding how best to split up and assign work”. It is the server that is responsible for distributing the workload and handling scheduling and tasks for the cluster. The “servants” are called nodes, which are “servers responsible for accepting and running workloads using local and external resources. “ To start everything a user submits a JSON or YAML plan, and the master server then runs the orchestration from there

Components of the master server that give it it’s brains

  • Etcd – uses to store configuration data that can be accessed by each of the nodes in the cluster.
  • Kube-apiservethe main management point of the entire cluster
  • Kube-controller-manager general service that has many responsibilities. Primarily, it manages different controllers that regulate the state of the cluster, manage workload life cycles, and perform routine tasks.
  • Kube-scheduler – The process that actually assigns workloads to specific nodes
    cloud-controller-manageract allows Kubernetes to interact providers with different capabilities, features, and APIs while maintaining relatively generic constructs internally.

While the master has all those components, we also cant forget about the kublet.
Its the main contact point for each node with the cluster group The kubelet service communicates with the master components to authenticate to the cluster and receive commands and work.

Checking for balanced parentheses using the stack

Posted on

So this problem, looks to see if there is a balance between round, curly, and square brackets.

First up lets discuss the the brackets:

  • ( ) Round Brackets
  • [ ] Square Brackets
  • { } Curly Brackets

Balance can be like so

  • (){}[]
  • ([{}])
  • (((((((((())))))))))

But this is unbalanced

  • ( ] ) [   notice closing square bracket without partner
  • ( [ ) ]   notice how the curved bracket closes before square bracket closes

If given a string of brackets, how can we tell if it validly pairs up? How can we solve this?

Use an array and every time we we don’t find a matching pair, add the element. If we do find a matching pair of the current element we are looking at and the previous one then pop the match.

What are you talking about?

  • (){}[]
  • ([{}])
  • (((((((((())))))))))

Looking at the above examples, you will eventually come across a pair of matching characters that are next to each other, and as we eliminate them, it frees up other pairs to be matched.

Please explain:

Ok, we will go over this step by step.

Lets loop over the string, removing matches as we find them. Only looking at last added element in array

  • string: ([{}]) . array or “stack” : []
  • string: ({ } ] )  array: []    no match in queue, add to queue
  • string: ( [ { } ] )  array: [ (  ]   no match in queue, add to queue
  • string: ( [ {] )  array: [ ( [  ]   no match in queue, add to queue
  • string: ( [ { } ] )  array: [ ( [ { ]   match in queue, remove from queue
  • string: ( [ { } ] )  array: [ ( [  ]   match in queue, remove from queue
  • string: ( [ { } ] )  array: [ (   ]   match in queue, remove from queue
  • string: ( [ { } ] )  array: [  ]   string traversed nothing in  queue

 

Here is a REPL in case you wanted to play with the code

Hadoop: An introduction

Posted on Updated on

This past week I attended a data conference where the functional piece behind a lot of the software that was being presented was Hadoop.”Hadoop is an open-source software framework for storing data and running applications” (1)

What makes Hadoop special is that it can be run on generic hardware, and the scalability is based on the amount of hardware you have, not the speed of your processor, or disk. Now to describe this in some details in layman’s terms: Imaging you are in your office with 100 employees. You have a laptop as does everyone else. If you want to processing something, you can only process as fast as your processor will go. OK this won’t cut it. You buy a huge Dell server with 16 processors, and each processor has 16 cores, giving you 256 threads. This set you back $250,000. Now you can really process a lot of data. But imagine you took all the laptops in your office and were able to cluster or chain them together. Dell XPS 13 laptops come with 2 cores, so if you chained all of them together, you could have a virtual “system: of 200 threads. At under $1000 each thats $200,000. At this point you might be thinking, ok where is the savings? The other system has more cores for not much more money. But now that we have our thinking going, what if we instead were looking at desktops, which we can buy at $399. Suddenly we have a “200 core” system at $40,000. (100 desktops x 399). $40,000 vs $250,000. Looking a lot better. Now lets scale this this up to a data center’s worth of hardware, and suddenly we are getting supercomputer performance and datacenter prices. This is just a generic example I thought of, but it shows why in scale Hadoop seems like a good choice. Like any infomercial I feel like I need to say “But wait! There more!”

Fault tolerance is built into Hadoop. FT is having the ability to not be affected by hardware failures. If a node, name given to a server or computer point, fails it does not mean that the jobs that were on that node are lost. Data and compute is distributed so jobs are redirected and data is stored on multiple points so the data on that node is also available elsewhere. An example that might make sense would be your iPhoto pictures. If you lose your phone, via iCloud they are still available on your PC, and even in the cloud if you paid for that plan. Your photos are fault tolerant, they don’t depend on your phone alone to exist.

Those are some of the ways that Hadoop makes sense to process big data, and why it exists. As I learn more, I hope I can teach you too.

ECMAScript 2017 padEnd() and padStart()

Posted on

Two new methods implemented in the 2017 version of Javascript are padEnd and padStart. Both methods pad a string with second string, until the result is of the desired indicated length. Given that the return value of padEnd and padStart are new Strings, and the original strings are unmodified, the methods a “pure”, as they do not cause side-effects.  As you can see from the examples below, this is true as str1 in the example is unmodified.

Screen Shot 2018-09-08 at 1.04.49 AM

Now the length of our example string, str1 is 17.  If the target length is less than the size of the primary string, the primary string will be returned. If the target length is less than the size of the primary string and the added string, the added string will be truncated to fit the target length. Example below:

Screen Shot 2018-09-08 at 1.14.54 AM.png

If your browser doesn’t have these methods because it doesn’t support ECMAScript 2017 you can actually write your own methods and add them to the String prototype, extending it. I changed the styling such that it can fit in a smaller picture, so heres an example of padEnd() implemented by me: below. See also the repl link if you would like to play with it

Screen Shot 2018-09-08 at 1.37.56 AM

Screen Shot 2018-09-08 at 1.42.44 AM.png

Binary: an introduction

Posted on

If I told you I would give you 10 dollars for a Five dollar bill, you probably would be excited. But if you received two dollars, you might be livid. “Where is the ten dollars you promised?” But I said 10 dollars not ten dollars. The 10 is binary. In binary 10 is 2. Are you confused? Don’t worry, it should make sense by the end

“ a binary number is a number expressed in the base-2 numeral system or binary numeral system, which uses only two symbols: typically 0 (zero) and 1 (one).”[1]

The number system we are used to is called decimal. The digits consist of 0,1,2,3,4,5,6,7,8,9 and are part of base ten notation. With binary, or base2 notation, the only digits are 0, and 1. While it may seem counterintuitive, every number that can be made with the decimal system can also be made in binary. Coincidentally this is how computers count and add digits, binary is the “native language” of computers.

What does zero to ten look like in binary?

  • Zero     0
  • One     1
  • Two     10
  • Three     11
  • Four    100
  • Five    101
  • Six    110
  • Seven    111
  • Eight    1000
  • Nine    1001
  • Ten    1010

 

Binary addition:

 

Addition in binary is just like decimal addition, adding from right to left, and carrying over carry values.

Lets go through an example: We will add Ten (1010) and eleven (1011)

  • Ten + Eleven = sum
  • Sum is currently 0
  • 1010 + 1011 : sum 0 : carry 0
  • Remember, Start with rightmost digits: Digits we worked on will be replaced with x
  • 1010 + 1011 = 0 + 1 = 1 : sum is 1 : carry is 0
  • 101x + 101x = 1 + 1 = 10: sum is 01 : carry is 1
  • 10xx + 10xx = 0 + 0 + carry(1) : sum is 101 : carry is 0
  • 1xxx + 1xxx = 1 + 1 = 10 : sum is 0101 : carry is 1
  • Sum is 10101
  • 10101 is 21 in decimal, addition complete!