Overview

1. Project background
2. Building an application
3. Configuring a server
4. Gathering a swarm
5. Demonstration

Project Background

1. PBS built a client-side Electoral College map
2. We built a real-time backend
3. We researched production readiness

Project Background: PBS's Offline Application

Project Background: Our Realtime Extension

Building the Application

The Client: Modularization

Intro to Socket.io

"Socket.IO aims to make realtime apps possible in every browser and mobile device, blurring the differences between the different transport mechanisms. It's care-free realtime 100% in JavaScript."

Intro to Socket.io (cont.)

Sometimes called "the jQuery for realtime applications."

Two components:

• socket.io-client: runs in the browser
• socket.io: runs on the server; serves the static JS socket.io-client files

The Client: Endpoint Parameterization

// Client source file
var defaultOptions = {
    // These values are expanded at build time.
    host: "",
    port: ""
};

The Client: Endpoint Parameterization (cont.)

// Build script (Grunt)
NODE_HOST: process.env.NODE_HOST || "127.0.0.1",
NODE_PORT: process.env.NODE_PORT || 8000,

// Inject NODE_HOST and NODE_PORT values into files as they are
// copied
processContent: function(content) {
  return content.replace(/\{\{\s*(NODE_HOST|NODE_PORT)\s*\}\}/g,
      function(match, varName) {
          return grunt.config("meta." + varName);
      }
  );
}

(Mostly) Avoiding fs.*Sync()

Generally a bad practice--it blocks the execution thread.

Acceptable during application initialization.

Advanced Socket.io usage: Learning the API

• Socket.io's examples (somewhat high-level)
• The wiki (incomplete)
• The source code (documents public API)

Advanced Socket.io usage: Dynamic connection handler

// One approach
socketServer.sockets.on("connection", function(socket) {
  if (app.isLive()) {
    connectLive.apply(socket, arguments);
  } else {
    offlineConnection.apply(socket, arguments);
  }
});

Advanced Socket.io usage: Dynamic connection handler

// Another approach
socketServer.sockets.on("connect", function(socket) {
  socketHandlers.connection.apply(socket, arguments);
});

// elsewhere...
socketHandlers.connection = connectLive;

Warning! Version 1.0 is on its way!

Configuring a Realtime Server

Security: Threat Model

• Brute force password guessing
• Private key theft
• Application logic compromise

Security: Precautions

• Disallow SSH password login
• Create dev user with sudo rights
• Create node user with limited rights

Technical requirements: Node.js

Technical requirements: Locking down dependencies

• NPM shrinkwrap
• npm-lockdown

Data collection tools: htop

Data collection tools: SAR

Gathering a Swarm

Client Simulator: Design

Goal

A script that simulates many people using your application simultaneously

Possible solutions

• Script a real browser to run your application (i.e. Selenium)
• Load your application in PhantomJS
• Strip out the network module and run that from Node.js

Client Simulator: Technical Considerations

• Using socket.io-client from Node.js (example)
• Supporting run-time configuration with optimist (example)
• Avoiding disk writes (even asynchronous) (example)
• Dispersing client connections (example)

Aside: Modifying upstream dependencies

Small-scale Demonstration

AWS*

"Amazon Web Services offers a complete set of infrastructure and application services that enable you to run virtually everything in the cloud: from enterprise applications and big data projects to social games and mobile apps."

Lucky for us, there's a Free usage tier.

AWS (cont.)

• Relax security considerations (instances are short-lived)
• Install your VCS to ease on-the-fly updates

Bees with Machine Guns

A utility for arming (creating) many bees (micro EC2 instances) to attack (load test) targets (web applications).

My awful hacked-up fork

"But... why?!"

We need a slightly more generalized approach to run our Node.js client simulator.

Demonstration

What's Next?

• stud and stud-proxy
• Socket.io 1.0 and Engine.io

Resources

• The Realtime Engine by Guillermo Rauch
• Source code
  • Socket.io
  • Engine.io
  • Bees with Machine Guns
  • PBS Map Center
• Related article series
  • Bocoup & PBS: Building and Scaling a Realtime Node.js Application
  • A Stress Testing Story
  • Stress Testing Procedure
  • Building a Server
• Related screencast series
  • Configuring AWS
  • Running the Simulation
  • Preparing a Server