Leverege Platform

Leverege Platform

  • Overview
  • JSDocs
  • REST API
  • Help

›Services

Platform Overview

  • Overview
  • Architecture
  • Core Concepts

Guides

  • Platform Setup
  • Building a Project
  • API UI

Services

  • API Server
  • Message Processor
  • Transponder
  • Reason
  • Scheduler
  • Messenger
  • Emailer
  • Rest Server
  • DB Curator

API

  • API
  • Interface
  • Project
  • System
  • Device
  • Blueprint
  • Collection
  • Child
  • History
  • Scenario
  • Script
  • Template
  • Timer
  • User
  • Users

Server Libraries

  • Log
  • Messages
  • Message Queue
  • Cache
  • Lock
  • Limit
  • Cluster Manager
  • Error
  • Exit

Utilities

  • Promise
  • Data Store
  • Factory
  • Path
  • Reasoner
  • Object Util
  • Array Util
  • String Util
  • Base62 Util
  • Value Cache
  • Validator
  • Lol

UI

  • API Redux
  • UI Builder
  • UI Elements
  • UI Redux
  • UI Mapbox Elements
  • UI Active Theme
  • UI Color Elements
  • UI Linear View Elements

DB Curator

DB-Curator is the service that is responsible for deleting old or stale data from databases after a specified period of time.

Structure

DB-Curator is structured as a set of long-running loops that run once per day at a specified time. The processes that clean up individual databases are independent, and run in parallel with each other.

Configuration

DB-Curator is overall configured with a small set of environment variables that determine which cleanup scripts run. By making the value of these variables false, one can turn off selected cleanup scripts.

ENV VariableDescription
SQL_CLEANUP_ENABLEDwhether or not the MySQL cleanup script is enabled
FB_CLEANUP_ENABLEDwhether or not the firebase cleanup script is enabled
PG_CLEANUP_ENABLEDwhether or not the PostgreSQL cleanup script is enabled

Additionally, certain env variables are shared between clean up scripts. The most important of these is 'SQL_KEEP_PERIOD' which tells both the MySql and Postgres scripts how long to go back before deleting data.

Firebase Configuration

ENV VariableDescription
FIREBASE_CLEANUP_CONFIGa JSON string specifying what paths in firebase to clean up, and how long to keep data

An example of the Firebase cleanup config is:

[
  {
    "listPath" : "rt/*/dev/*/geolog",
    "keepTime" : 604800000,
    "itemTimePath" : "time"
  },
  {
    "listPath" : "rt/*/dev/*/events/*",
    "keepTime" : 604800000,
    "itemTimePath" : "time"
  }
]

The config specifies locations (as an array) in firebase of time ordered lists to be cleaned up using the following three keys:

KeyDescription
listPathpath in firebase (including *'s) to the list or set of lists to be cleaned up
keepTimelength of time to keep data before it is deleted (ms)
itemTimePathsince each list or set of lists should be comprised of similar objects, this specifies the time key of each item that can be used to determine the age of the object

Postgres Configuration

ENV VariableDescription
PG_POOL_CONNECTION_LIMITmax number of connections to make to postgres
PG_HOSThost the database is located at
PG_PORTport the database is located at
PG_USERpostgres user name
PG_PASSWORDpostgres password
PG_DATABASEdatabase to connect to

MySQL Configuration

ENV VariableDescription
SQL_CONNECTION_LIMITmaximum connections to make to the sql server
SQL_HOSThost the db is located at
SQL_PORTport the db is located at
SQL_USERusername to login with
SQL_PASSWORDpassword to login with
SQL_CAserver's CA if it has ssl enabled
SQL_CLIENT_CERTclient's cert if the server has ssl enabled
SQL_CLIENT_KEYclient's key if the server has ssl enabled
← Rest ServerAPI →
  • Structure
  • Configuration
    • Firebase Configuration
    • Postgres Configuration
    • MySQL Configuration
Leverege Platform
Docs
Overview
Connect
FacebookLinkedInTwitter
Facebook Open Source