Twelve-Factor Apps In node.js

About Me

• Independent Consultant
• Full-stack web applications
• node.js specialist
• https://peterlyons.com

What is Twelve-Factor?

What is Twelve-Factor?

• 12factor.net
• A methodology for application lifecycle management
• Based on Heroku's experience running 1000s of applications
• Optimized for the cloud era: IaaS, SaaS, PaaS, scale out
• Embodies opinions and best practices
• Addresses "systemic problems"
• not the only valid approach
• not appropriate for all applications or organizations

Pre-PaaS Genisis of Twelve-Factor

Deployment and Operations Before PaaS

• sysadmins + ssh + bash
• undocumented systems, tribal knowledge
• manual repetition of procedures
• error prone
• easy to get into inconsistent state
• scary reboots and rebuilds
• precious golden servers
• relatively tight coupling between the app and the underlying OS
• high variance between dev/test/prod environments
• high variance across companies/applications

Platform as a Service (PaaS)

• takes off around 2008 (Google App Engine)
• How does this effect application lifecycles?

• as PaaS product category blooms, industry needs to converge on automatable conventions

• need standard workflow

• get latest source

• create build artifact

• deploy to servers

• restart services

The Twelve Factors

1. Codebase

1. Codebase

• Use git or $SCM

• 1 to 1 mapping repo to app

• have at least distinct prod and stage deploments

• 1 repo containing multiple apps: nope

• 1 app built from multiple repos: nope

• node.js specifics

• extract shared libraries to separate npm repos

• use private git repo URLs or npm Enterprise

2. Dependencies

2. Dependencies

• use npm package.json properly
• Incorrect usage of npm accounts for 25-50% of issues Heroku sees with their node.js users
• npm config set save-exact=true
• npm config set save=true
• npm config set save-prefix=
• specify exact versions

2. Dependencies…

• use shrinkwrap
• never install with npm -g even if README says to
• no system-level subprocesses
  • docker containers can help with this
  • This is highly dubious in the long term and espouses the "Inner Platform Effect" antipattern
  • the HTTP implementation in cURL is almost certainly far superior to the one in $YOUR_RUNTIME

3. Config

3. Config

• app loads from OS environment variables

• In systemd, use EnvFile=

• don't group into named "environments"

• production, development, staging, etc

• node-formancan help transition away from this

• npm module: config3

4. Backing Services

• treat as loosely coupled
• connect via config params
• switch between local and third-party providers with config changes but no code changes
• sadly, most node db libraries do not make it easy to detect and gracefully handle non-working db connections

6. Processes

systemd service config file

• Install to /etc/systemd/system/mynodejsapp.service

[Unit]
Description=My node.js App

[Service]
User=mynodejsapp
Group=mynodejsapp
WorkingDirectory=/opt/mynodejsapp
EnvironmentFile=/etc/mynodejsapp/config
Environment=NODE_ENV=production
ExecStart=/usr/bin/node cluster.js
Restart=always

[Install]
WantedBy=multi-user.target

upstart config file (Debian/Ubuntu before 15.04)

• Install to /etc/init/mynodejsapp.conf

description "mynodejsapp"
start on filesystem and started networking
respawn
console log
chdir /opt/mynodejsapp
setuid mynodejsapp
setgid mynodejsapp
env PATH=./node_modules/.bin:./node/bin:/usr/bin
env NODE_ENV=production
exec app/server.js

6. Processes

• Don't run as root
• don't daemonize
• Don't use a node-specific process supervisor
• Use systemd or docker restart policies
• local filesystem only as temporary, atomic operation working space as needed

7. Port Binding

• straightforward
• for web apps, FQDN can come from config

8. Concurrency

8. Concurrency

• scale out with the shared-nothing process model
• design app as distinct worker/process types
• local state is an antipattern Heroku commonly sees

9. Disposability

• robust queues
• crash-only

10. Dev/prod parity

• Don't use "lightweight" dev variants of backing services (sqlite3 etc)
• It's a Vagrant/Docker party and everyone's invited
• Dev: OSX + node.js + mynodejsapp, Docker + PostgreSQL
• Test/Stage/Prod: Docker + node.js + mynodejsapp, Docker + PostgeSQL

11. Logs

• bole, bunyan, winston
• newline-delimited JSON to stdout
• let the environment deal with rotation, centralization
• systemd/journald handle it automatically
• upstart "console log" handles it
• svlogd from runit
• multilog from daemontools

12. Admin Processes

• stick them in a bin directory

Other Industry Trends

• Immutable Infrastructure
• pets -> cattle
  • http://www.slideshare.net/randybias/architectures-for-open-and-scalable-clouds
  • https://blog.engineyard.com/2014/pets-vs-cattle

Merits of Commercial Service

• Some are worth their weight in gold
• Some are long-term tax on lack of knowledge/experience
• Commercial offerings are often superior to beginner/intermediate in-house approaches, but inferior to expert in-house systems

Sample App Tour of mjournal

1. Codebase: code
2. Dependencies: code
3. Config: code
4. Backing Services: code
5. Build, Release, Run: code
6. Processes: code

7. Port Binding: code

8. Concurrency: (Oops!) code

9. Disposability: code

10. Dev/prod parity: code

11. Logs: code

12. Admin Processes: code

Thank You

Hunter Loftis (heroku node build pack maintainer) for review/feedback

References

• 12factor.net
• docker containerization
• Heroku Docker Plugin
• throng module for cluster worker management
• config3

• StrongLoop node.js Deployment Best Practices
• Immutable Infrastructure is the Future
• My Post About Avoiding npm -g
• Definition and purpose of a staging environment
• Environment Variables Considered Harmful

The End

peterlyons.com