developing apps using perl

Developing appsusing Perl

Anatoly SharifulinYAPC::Russia 2012

Hello, World!

«Happy Perl developer»about me by Andy Shitov

I program in Perl lessBut it doesn't mean I love it less :-)

JFDI became JFTIJust F*cking Talk about It

Success-storyof one app

And how Perl helped me/us with it

By us I mean Applifto

www.applifto.com

Our applications

Audience polling

DLTTR.com

«Why?» and «For what?» in details

http://www.slideshare.net/sharifulin/ss-12313103

A powerful reasonfor me

I was moving from LiveJournal to Posterous, used transfer scripts and changed Twitter and Facebook

crossposting settings when started to get replies like 'Tolya! Stop it! Enough!'

Current analogues are nice... not!

Slow, awkward, plain, trouble working:TwitWipe, TwitCide, Delete My Tweets, ...

«Stop managering, let’s start programming»

And have a talk at #yr2012 :-)

Approach and solutions

Main aims for me

• App must delete tweets quickly

• Appropriate deleting

• Cross-platform app: iOS, Android, Web, ...

Server API— hello, Perl!

All logic — on our server API (not Twitter API),clients is simple interface

Getting to know Twitter API

Little experience:

• Getting user profiles after login through Twitter on websites

• Auto-posting to Twitter with Net::Twitter(::Lite)

• AnyEvent::Twitter::Stream for real-time search (hey, @miyagawa!)

Blocking wayTask can be solved with Net::Twitter(::Lite)

LWP::UserAgent::POE Transport may be changed in Net::Twitter(::Lite)

NO

Idea — 2 separate asynchronous queues

Get timeline and delete tweets using REST Twitter API and asynchronous requests

This has to work fast!

Authorization and OAuth

Net::OAuth::AllWe've written our OAuth module long time ago,

supports all OAuth versions (1.0, 1.0A, 2.0)

Net::OAuth::Allhttps://github.com/likhatskiy/Net-OAuth-All

my $oauth = Net::OAuth::All->new( consumer_secret => $conf->{consumer_secret}, consumer_key => $conf->{consumer_key },

token => $data->{access_token }, token_secret => $data->{access_token_secret},);

$oauth ->via('GET') ->protected_resource_url('https://api.twiiter.com/...’) ->put_extra( include_rts => 'true', user_id => $user->{id}, count => 200, ) ->request('protected_resource');$oauth->to_header;$oauth->url; # $oauth->url_with_extra;

Patched the moduleUnauthorized user:

limit of 150 requests per hour from one IP, not 350 requests per hour from one user

Asynchronous requests

I use MojoliciousNot a secret at all :-)

Mojo::UserAgentMojo::IOLoopGood asynchronous HTTP client

my $delay = Mojo::IOLoop->delay;for (@$tasks) { ... $delay->begin; $ua->get($url => {'Authorization' => $h} => sub { my ($ua, $tx) = @_; ... $delay->end( ... ); });}

say $delay->wait;

Server API

Server APIAPI supports GET and POST requests,

JSON transmission format,response codes 200 and 50x.

Server APIStarman + Mojolicious

Mojolicious::Plugin::ApiHelpers

Server APIhttp://api.dlttr.com

package App::Api;use App::Base -controller, with => [ 'App::Task', 'App::User' ];

sub error { shift->api_error(code => 1) }sub any { shift->api_error(code => 5) }

...

1;

my $api = $r->route('/api')->to('api#', api => 1);$api->route('/:action', action => qr/login|oauth/)->to;

my $apiu = $api->bridge->to('#auth', sign => 1);$apiu->bridge('/task/new')->to('#task_check') ->route->to('#task_new');

$api->bridge('/')->to('#auth', maybe => 1, sign => 0) ->route->to('#hello');

10 methods,123 tests and documentation

Most methods require user authorization and check sign (api_id + secret)

Test::MojoTest::More

Testing API methods access, input-output data, signature check and etc.

Test::MojoTest::More

A real task from a test user

All methods are coveredWithout Mock-objects, no check

if a correct cluster of tweets have been deleted

Too laborious and needlessA real cycle cannot be tested:

posting tweets – deleting – check

$t->get_ok("$url/") ->status_is(200) ->header_is('Access-Control-Allow-Origin', '*') ->json_content_is({hello => 'Hello DLTTR!'});

$t->get_ok(sign get => "$url/task/new") ->status_is(200) ->json_content_is({error => {msg => 'User authorization failed', code => 2}});

Too risky to test on your own account :-)

— You don't care about you Twitter acc, do you?— Yep!— So I can delete all your tweets?— Nooooooooooooooo!!!

Simple debug reguests

To check the quality of client work requests/responses with before_dispatch + after_dispatch may be used

Three queues

1. Queue for timelinesSearch and filters on tweets ID to delete,

paging with since_id and max_id

1. Queue for timelinesMax. 200 tweets per request,

getting timeline one by one per user, Rate Limit check

«Limits» by TwitterNo access to all tweets,

only the latest ~3200 or starting from any exact date, counter is not on zero and etc.

2. Queue for deleting tweets

Deleting with 'clusters' of 200 tweets, no limits. «Buster» or «Cannon» :-)

2. Queue for deleting tweets

Error processing — Twitter API error doesn't always mean a failure

3. Queue for Push notify

On iOSNet::APNS::Persistent (also AnyEvent::APNS),

to check if tokens are valid — Net::APNS::Feedback

On AndroidWWW::Google::C2DM and WWW::Google::ClientLogin

3. Queue for Push notify

Database

So easy — mysqlStoring and task orders,

just to relax :-)

2 finer points:

1. Create correct indexFor queues

2. Disable request caching

select SQL_NO_CACHE * from task where ...

Site and web-version

www.dlttr.com

www.dlttr.com

Site and web-version

• Starman + Mojolicious

• Mojolicious::Plugin::I18N2— my fork of I18N plugin

• Mojolicious::Plugin::OAuth— login through Net::OAuth::All

• Mojolicious::Plugin::ShareHelpers

• Mojolicious::Plugin::UtilHelpers

Admin and statistics

«Big Brother»All details on users, number of tasks, feedbacks

and real-time statistics

Statistics on users

Users wall

«Ordinary» apps don't have statistics like that

1M+ tweets deleted,3K+ users,

20% make purchaseFor 2 months

Promotion

Promotion on TwitterFollowing, favorites, replies and search —

Net::Twitter(::Lite)

But they blocked me three times already :-)

Promotion on Twitter

Summary

Positive

• Easy solution using pure Perl(work time is about 30 hours)

• Works fast, really fast

• Right delete tweets

• Any changes, updates, new services— everything's on the server, no hard

• Real-time statistics

Only one negative:Server's down — the app doesn't work,

users become mad, low rates and spell curses :-)

Check out DLTTR!http://dlttr.com/app http://dlttr.com/android

www.dlttr.com

«Seems like a simple app but in fact it's awesome

and in Perl»

P. S.

At #BarCampKrrI had a talk

«Developing app from idea to release»

(Perl wasn't mentioned)

But everything came down to... Perl

«What language do you use?», «But why?»,

«Is it difficult to find programmers like those?» :-)

use Perl or die;

Thanks!Anatoly SharifulinYAPC::Russia 2012