job queue in b2b with akka
DESCRIPTION
Job queue in b2b with AkkaTRANSCRIPT
![Page 1: Job queue in b2b with Akka](https://reader034.vdocuments.us/reader034/viewer/2022051312/5476d202b4af9f933e8b472d/html5/thumbnails/1.jpg)
Job Queue in B2Bwith Akka
long versionYasuki Okumura
![Page 2: Job queue in b2b with Akka](https://reader034.vdocuments.us/reader034/viewer/2022051312/5476d202b4af9f933e8b472d/html5/thumbnails/2.jpg)
● Shanon, Inc. - Marketing company
● Scala developer for 2 years
twitter: @busterdayogithub: https://github.com/buster84
About me
![Page 3: Job queue in b2b with Akka](https://reader034.vdocuments.us/reader034/viewer/2022051312/5476d202b4af9f933e8b472d/html5/thumbnails/3.jpg)
Shanon hiring Scala developers
![Page 4: Job queue in b2b with Akka](https://reader034.vdocuments.us/reader034/viewer/2022051312/5476d202b4af9f933e8b472d/html5/thumbnails/4.jpg)
● Job Queueing manager● https://github.com/Shanon/Malba
Malba
![Page 5: Job queue in b2b with Akka](https://reader034.vdocuments.us/reader034/viewer/2022051312/5476d202b4af9f933e8b472d/html5/thumbnails/5.jpg)
● Most problematic thing in B2B is job order.● Most of B2B service cases, normal FIFO or
priority Queue don’t fit their business model.
● We need our own Custom Queue.
Motivation
![Page 6: Job queue in b2b with Akka](https://reader034.vdocuments.us/reader034/viewer/2022051312/5476d202b4af9f933e8b472d/html5/thumbnails/6.jpg)
Example - Email service
1. A attempt to send 100,000 emails.2. B attempt to send 100 emails.
Does B have to wait until finishing to send 100,000 emails?
![Page 7: Job queue in b2b with Akka](https://reader034.vdocuments.us/reader034/viewer/2022051312/5476d202b4af9f933e8b472d/html5/thumbnails/7.jpg)
Balancing Queue
A’s 100,000 emails
FIFO
B’s 100 emails
1
32
4
100,001
100,100
Balancing Queue
1 2
100,100
200
4
199
3
![Page 8: Job queue in b2b with Akka](https://reader034.vdocuments.us/reader034/viewer/2022051312/5476d202b4af9f933e8b472d/html5/thumbnails/8.jpg)
● Breadth-first search -> FIFO Queue● Depth-first search -> FILO Queue● A* search -> Priority Queue
Inspired by Graph search
What makes difference among the algorithms is Queue.
![Page 9: Job queue in b2b with Akka](https://reader034.vdocuments.us/reader034/viewer/2022051312/5476d202b4af9f933e8b472d/html5/thumbnails/9.jpg)
● Premium users can send mails faster than normal users.
● It is guaranteed that sending less than 1,000 mails finish in 1 hour.
Other cases
![Page 10: Job queue in b2b with Akka](https://reader034.vdocuments.us/reader034/viewer/2022051312/5476d202b4af9f933e8b472d/html5/thumbnails/10.jpg)
How we use
![Page 11: Job queue in b2b with Akka](https://reader034.vdocuments.us/reader034/viewer/2022051312/5476d202b4af9f933e8b472d/html5/thumbnails/11.jpg)
1. AddTaskhttps://github.com/Shanon/Malba/blob/master/Malba-protocol/src/main/scala/jp/co/shanon/malba/worker/MalbaProtcol.scala#L9-L17
2. AddWorkerhttps://github.com/Shanon/Malba/blob/master/Malba-
protocol/src/main/scala/jp/co/shanon/malba/worker/MalbaProtcol.scala#L151-L158
3. GetTaskhttps://github.com/Shanon/Malba/blob/master/Malba-protocol/src/main/scala/jp/co/shanon/malba/worker/MalbaProtcol.scala#L56-L63
Structure + Protocol
![Page 12: Job queue in b2b with Akka](https://reader034.vdocuments.us/reader034/viewer/2022051312/5476d202b4af9f933e8b472d/html5/thumbnails/12.jpg)
![Page 13: Job queue in b2b with Akka](https://reader034.vdocuments.us/reader034/viewer/2022051312/5476d202b4af9f933e8b472d/html5/thumbnails/13.jpg)
![Page 14: Job queue in b2b with Akka](https://reader034.vdocuments.us/reader034/viewer/2022051312/5476d202b4af9f933e8b472d/html5/thumbnails/14.jpg)
![Page 15: Job queue in b2b with Akka](https://reader034.vdocuments.us/reader034/viewer/2022051312/5476d202b4af9f933e8b472d/html5/thumbnails/15.jpg)
Demo
![Page 16: Job queue in b2b with Akka](https://reader034.vdocuments.us/reader034/viewer/2022051312/5476d202b4af9f933e8b472d/html5/thumbnails/16.jpg)
https://github.com/typesafehub/activator-akka-distributed-workershttp://letitcrash.com/post/29044669086/balancing-workload-across-nodes-with-akka-2
You can compare Malba with above.
Strongly inspired by
![Page 17: Job queue in b2b with Akka](https://reader034.vdocuments.us/reader034/viewer/2022051312/5476d202b4af9f933e8b472d/html5/thumbnails/17.jpg)
● CI environment● Make and Publish Packages● Support HTTP● Tests● Documents● Take snapshots● GUI - using PersistentView
TODO
![Page 18: Job queue in b2b with Akka](https://reader034.vdocuments.us/reader034/viewer/2022051312/5476d202b4af9f933e8b472d/html5/thumbnails/18.jpg)
We welcome your PR or comments.https://github.com/Shanon/Malba
Thank you