Couchbase Connect Fall 2017 Demo

Application code for Couchbase Connect SF Fall 2017 demo talk.

Credits

This project includes significant contributions from others.

The web client is based on CoPilot and is used here under terms of the MIT License.

The Android source includes a port of Jasonette and is included here under terms of the MIT License.

Images used in the web client are copyright their respective owners. Attributions are listed under web/client/static/img/README.md.

Other copyrights may apply. Inclusion in this work does not imply additional rights beyond those granted by the copyright holders.

This overall project, as applicable, is released under terms of the Apache 2.0 License. See LICENSE for details.

Instructions

The instructions come in a few forms.

There are several scripts for both Unix command line and Docker-based installations to simplify setup. Instructions based on using the scripts to come.

There’s a short form of the instructions meant for easy copy and paste.

There’s a longer form description that breaks down the steps for better understanding.

Finally, you can find a video walking through the entire setup using the link below.

Couchbase Connect SV 2017 Demo Setup Walkthrough

Tl;dr version

This will show how to configure the demo mostly using the command line.

(N.B. Command line examples are given for a Mac using Bash.)

Clone Repository and Set Working Directory

git clone https://github.com/couchbaselabs/connect-fall-2017-demo.git
cd connect-fall-2017-demo

Couchbase Server

Download and install Couchbase version 5.5 or later. Add the Couchbase Server tools directory to your command path.

export PATH="$PATH:/path/to/install/Couchbase Server.app/Contents/Resources/couchbase-core/bin"

Start in the top level directory of the project (connect-fall-2017-demo).

Run Couchbase Server. The following assumes you’re running on localhost. Configure the node via the command line.

# Basic memory and service configuration
couchbase-cli cluster-init --cluster couchbase://127.0.0.1 --cluster-name cluster \
  --cluster-username Administrator --cluster-password password \
  --services data,index,query,fts,analytics,eventing --cluster-ramsize 256 --cluster-analytics-ramsize 1024 \
  --cluster-index-ramsize 256 --cluster-fts-ramsize 256 --index-storage-setting default
# Main bucket creation
couchbase-cli bucket-create --cluster couchbase://127.0.0.1 -u Administrator -p password \
  --bucket health --bucket-type couchbase --bucket-ramsize 100
# Role-Based Access Control
couchbase-cli user-manage --cluster couchbase://127.0.0.1 -u Administrator -p password \
  --set --rbac-username admin --rbac-password password \
  --rbac-name "J. R. Admin" --roles "Admin" --auth-domain local

Wait for the node to warm up (about 15 seconds or so). Configure the rest of what Couchbase Server needs.

# Indexes for query optimization
cbq -u admin -p password -q --script="CREATE INDEX \`resource-idx\` ON health(resourceType, id);"
cbq -u admin -p password -q --script="CREATE INDEX \`observation-idx\` ON health(subject.reference, issued DESC, valueQuantity.\`value\`)"
cbq -u admin -p password -q --script="CREATE INDEX \`location-idx\` ON health(type.coding[0].code) WHERE resourceType = 'Location';"
# cURL site whitelist
curl -X POST -u admin:password -d "@scripts/config/curl" http://127.0.0.1:8091/settings/querySettings/curlWhitelist
# Full-Text Search index
curl -u admin:password -T "scripts/config/fts-index.json" http://127.0.0.1:8094/api/index/diagnosis
# Eventing Service meta-data bucket creation
couchbase-cli bucket-create --cluster couchbase://127.0.0.1 -u Administrator -p password \
  --bucket eventing --bucket-type couchbase --bucket-ramsize 100
# Eventing Service function
curl -X POST -u admin:password -d "@scripts/config/eventing" http://127.0.0.1:8096/api/v1/functions/monitor
# Analytics Service indexes for query optimization
cat scripts/config/analytics | while read query
do
  curl -u admin:password --data-urlencode "statement=${query}" http://127.0.0.1:8095/analytics/service
done
# Load sample data
for file in data/*.json
do
  cbimport json -c couchbase://127.0.0.1:8091 -d file://${file} -g '%id%' -f lines -b health -u admin -p password
done

Web Client and Server

Start in the top level directory of the project (connect-fall-2017-demo).

# Build Vue.js web client
cd web/client
npm install
npm run build
# Build and run Node.js web server
cd ../server
npm install
cat > .env <<EOF
UA_APPLICATION_KEY='<your app key>'
UA_APPLICATION_MASTER_SECRET='<Your app master secret>'
CLUSTER='couchbase://localhost'
CLUSTER_USER='admin'
CLUSTER_PASSWORD='password'
CLUSTER_CBAS='localhost:8095'
PORT=8080
EOF
npm start

Sync Gateway

Start in the top level directory of the project (connect-fall-2017-demo).

/path/to/couchbase-sync-gateway/bin/sync_gateway sync-gateway/cc-2017-sg-config.json

Android Mobile Application

Open mobile/android/CBCHealth in Android Studio to build the application.

The Android mobile app uses Urban Airship for push notifications. If you want to include this feature, you must fill out the Urban Airship configuration with your own keys. See mobile/android/CBCHealth/app/src/main/assets/airshipconfig.properties.sample.

If you don’t want to include push notifications, remove the following line from the application’s AndroidManifest.xml file.

<meta-data
  android:name="com.urbanairship.autopilot"
  android:value="com.couchbase.mobile.notifications.Autopilot" />

Detailed Version

This is a work in progress.

On the machine that will host the web server

Install node. Note the server requires version 7 or higher. I recommend using nvm to manage Node versions if you have an existing installation. (The nvm installation guide can be found here.

Clone the repo

git clone https://github.com/couchbaselabs/connect-fall-2017-demo.git

Configuring the server

cat > .env
UA_APPLICATION_KEY='<your app key>'
UA_APPLICATION_MASTER_SECRET='<Your app master secret>'
CLUSTER='couchbase://localhost'
CLUSTER_USER='<username>'
CLUSTER_PASSWORD='<password>'
CLUSTER_CBAS='localhost:8095'

Running the server

Build requires a package to compile the native part of the Couchbase Node client. Install if needed.

npm install -g node-gyp

You may also need to install the native compilation tools (e.g. g++). On Redhat

yum group install "Development Tools"

In the directory demo/web/server install Node packages.

npm install

Run the server.

node ./bin/www

Or, for simple crash resilience, run a script.

#! /usr/bin/env bash

while true
do
  node ./bin/www
done

To prevent some systems from killing the process when you log out, run with nohup, like this.

nohup ./server.sh < /dev/null >& server.log &

Configuring the client

Under src/config in the client code, update serverURI in the index.js file to point to your web server.

Building the client

Shift to the directory demo/web/client. Install the Node packages.

npm install

Decide if you want to run the production version or development version. The development version supports hot reloading, but currently requires running a separate development server.

To use the development server:

npm run dev

You should find the pages served up on localhost:8080.

To run a production version:

npm run build

You should find the pages served on localhost:3000

Enabling push notifications

The Node web server reads configuration parameters for Urban Airship from the shell environment. In the shell, before running the server, export the keys as follows.

export UA_APPLICATION_KEY=<your application key>
export UA_APPLICATION_MASTER_SECRET=<your application master secret>

These keys come from your Urban Airship project.

Couchbase Server

Data

The file augment-data.json contains records hand written to work with the demo. To add these to a bucket, use (e.g.)

cbimport json -u admin -p password -b health -c couchbase://127.0.0.1:8091 -d file://augment-data.json -g '%id%' -f lines

On Macs look for cbimport in /Applications/Couchbase\ Server.app/Contents/Resources/couchbase-core/bin/

Whitelisting sites for curl

The curl functionality in N1QL requires sites to be white/black listed. For this application, whitelist the Google geocoding endpoint by creating

/opt/couchbase/var/lib/couchbase/n1qlcerts/curl_whitelist.json

with contents

{
  "all_access":false,
  "allowed_urls":["https://maps.googleapis.com/maps/api/geocode/json"]
}

Indexes

Several queries examine resourceType and id:

CREATE INDEX `resource-idx` ON health(resourceType, id);

Mapping hospitals queries on location resources and specific type codes:

CREATE INDEX `location-idx` ON `health`(type.coding[0].code) WHERE resourceType = 'Location';

Monitoring incoming observations from our select patient:

CREATE INDEX `observation-idx` ON `health`((`subject`.`reference`),`issued` DESC,(`valueQuantity`.`value`))

Full text search:

The full text search index definition can be found in demo/models/fts-index.json. Load it with something like this.

curl -T fts-index.json http://admin:password@localhost:8094/api/index/diagnosis

Installing Couchnode from GitHub

To install from the latest

npm install --save git+https://git@github.com/brett19/couchnode.git

To pin the installation to a specific commit

npm install --save git+https://git@github.com/brett19/couchnode.git#dba79ef33b1f4e7d5e88906538624c65caf3d841