project meniscus

Project Meniscus

Simplex Internet / Research Institute / Jeong, Ki-Chul

2013. 4. 29

Meniscus?

• LAAS(Logging-as-a-Service) Backend API

• Python based system

• Event - collection, transit, processing

• Large-scale

• Cloud Logging

SEE THE ANIMATED OVERVIEW

APIS

Tenant API

• Tenant = {Host, Event Producer, Profile}

• A Tenant = A Consumer

• Event Producer = APP or Service

• Profile = [Event Producer, …]

• Host = Node , link to a profile(1:1)

Tenant API

• Requests to coordinator worker

• Tenant, Host, Event Producer, Profile

=> CRUD Available

• has a TOKEN

=> validate, reset a token

Correlator API

• Requests

To a Correlator worker server

From a Consumer server

via http(s)

with a Message token(Tenant token)

• Stores a collection named “logs” in Mongo

Coordinator API

• Pairing

- registers a worker

• Configuration

– requests a worker’s configuration

• Status

– updates a workers status

Status API

• Worker Status API (PUT, Manually)

- System Load Average

- Disk Usage

• Coordinator Status API

- grid status

- individual worker status

Broadcaster API

• What’s Brodcaster API?

I don’t know well!

• Communicates with coordinator worker

server.

• New Feature!

GETTING STARTED

Server deployment example

Host1

• Coordinator

• MongoDB

Host2

• Correlator

Host3

• Storage

HOST1

• Creating a development environment

[LINK]

• Install MongoDB

[LINK]

uWSGI.ini

meniscus/etc/meniscus/meniscus.conf

Run uwsgi daemon!!

# uwsgi uwsgi.ini

HOST2

• Creating a development environment

[LINK]

• Run uwsgi daemon!!

Default Persona : pairing

CURL Request

• Attempts pairing with coordinator

• Becomes correlation worker after automatically restart

HOST3

• Creating a development environment

[LINK]

• Run uwsgi daemon!!

Default Persona : pairing

CURL Request

• Attempts pairing with coordinator

• Becomes store worker after automatically restart

COMPLETED FOR SETTING UP

Logging a data

• Already MUST register a tenant

• Request to correlator server(host2)

Flow

Correlator

(host2)

Storage

(host3)

MongoDB

(host1)

Conclusion

• Well-made backend APIs for logging

(Good architecture)

• Distributed workers processing

• Diverse Persona(worker)

• In early development

• Nobody in its community

(Stack Overflow, Google Groups)

Not yet

• Data Store – Only MongoDB

Hadoop / HDFS / Hbase

Elastic Search

• Perfect documentation

need more examples

need exact examples

The thing we can now do with meniscus is

JUST to create log data into mongodb via

restful api and to fetch log data from

mongodb.

THANKS