developmental aids whitepaper 9-1-2017 › media › files › white_paper › ... · apache sentry...

DEVELOPMENTAL AIDSHOW THIRD PARTY SOFTWARES ARE MAKING LIFE EASIER IN THE DEVELOPMENT WORLD

IntroductionAgile projects are executed at break-neck speed nowadays. Finishing the

project in time and within budget without compromising on quality has

become a bigger challenge than in the past. This is compounded by issues

like design complexity and the difficulties of confining development effort to

time-boxed sprints. This is where the advent of cool aids in the form of a host

of third party software has proved to be a big boon for project teams. So let

us take a look at some of the prominent aids.

AWESOME

DEVELOPMENTAL

AIDS

www.sayonetech.com 02

Developmental Aids

Crashlytics

Google Analytics

Deeplinking

Papertrail

Apache Bench

Logentries

New Relic

Index

04

05

07

08

09

10

11


Developmental Aids

Apache Sentry

Apache JMeter

Amazon Web Services

Mailgun

SendGrid

Campaign Tracker

Summary

12

13

15

17

18

19

20

CrashlyticsCrashlytics is a popular software development kit (SDK) owned by Twitter. It helps iOS and Android developers to pinpoint issues causing their app’s

instabilities. With Crashlytics’ service, which includes its own online dashboard, crash logs are distilled down; highlighting the real issues, detailing the device’s

state at the time of crash (software version, orientation, model, etc.) and it even shows developers the exact line of code that the app crashed on. Crashlytics is

the most powerful, yet lightest weight crash reporting solution providing reliable performance SDK on both iOS and Android, Crashlytics provides deep and

actionable insights. Over 2 billion active devices have Crashlytics installed in them. The company serves thousands of companies, including Yelp, Kayak,

PayPal, Walmart, Groupon, Waze, Blackboard, Path, OpenTable, Domino’s, Expedia and Square.


Developmental Aids

Google AnalyticsGoogle Analytics is a freemium web analytics service offered by Google that tracks and reports website traffic. Google Analytics shows high-level,

dashboard-type data for the casual user and more in-depth data further into the report set. Google Analytics analysis can identify poorly performing pages

with techniques such as funnel visualization, where visitors came from (referrers), how long they stayed and their geographical position. It also provides

more advanced features, including custom visitor segmentation.

Google Analytics e-commerce reporting can track sales activity and performance. The e-commerce report shows a site's transactions, revenue, and many

other commerce-related metrics. A user can have 100 site profiles. Each profile generally corresponds to one website. It is limited to sites which has traffic

of fewer than 5 million page views per month unless the site is linked to an AdWords campaign.

Google Analytics includes Google Website Optimizer, rebranded as Google Analytics Content Experiments.

Google Analytics Cohort analysis feature helps to understand the behavior of component groups of users apart from your user population. It is very much

beneficial to marketers and analysts for successful implementation of Marketing Strategy.


Developmental Aids

Developmental Aids


Google Analytics is implemented with page tags, called the Google Analytics Tracking Code, which is a snippet of JavaScript code that the website owner

adds to every page of the website. The tracking code runs in the client browser when the client browses the page (if JavaScript is enabled in the browser)

and collects visitor data and sends it to a Google data collection server as part of a request for a web beacon.

The tracking code loads a larger JavaScript file from the Google web server and then sets variables with the user's account number. The larger file

(currently known as ga.js) is typically 18 KB. The file does not usually have to be loaded, however, due to browser caching. Assuming caching is enabled in

the browser, it downloads ga.js only once at the start of the visit. Furthermore, as all websites that implement Google Analytics with the ga.js code use the

same master file from Google, a browser that has previously visited any other website running Google Analytics will already have the file cached on their

machine.

In addition to transmitting information to a Google server, the tracking code sets a first party cookie (If cookies are enabled in the browser) on each visitor's

computer. This cookie stores anonymous information, called the Client Id. Before the launch of Universal Analytics, there were several cookies storing

information such as whether the visitor is new of had been to the site before, timestamp of the current visit, and the referrer site or campaign that directed

the visitor to the page (e.g., search engine, keywords, banner, or email).

Google Analytics

DeeplinkingIn the context of the World Wide Web, deep linking is the use of a hyperlink that links to a specific, generally searchable or indexed, piece of web content on a

website (e.g., http://www.sayonetech.com/services/web-applications/), rather than the website's home page (e.g., http://www.sayonetech.com/). Branch.io

refers to the open source mobile linking service from Branch Metrics. The Branch Metrics SDK enables developers to integrate their apps with deep linking,

contextual deep linking, organic, paid and referral attribution, and app-to-app communication. Use the branch.io tag for questions relating to Branch's Web,

iOS, and Android SDKs, Branch Links and registering Webhooks.


Developmental Aids

PapertrailPapertrail is a cloud-based, log-management service. Machines’ log files have always been important for troubleshooting, but are seeing increased interest

with the advent of big data because companies have realized they can draw business-level insights from log data, too. Papertrail is gaining popularity at a time

when organizations are embracing both cloud services and big data tools. Papertrail offers a place to store, search and analyze log files from a number of

different sources for a low monthly fee. The service can handle a variety of operating system logs, app server requests, database queries, and router logs.

Papertrail’s features include a graphical dashboard, search capabilities via command line or RESTful API, and email alerts. Papertrail is hosted on Amazon’s

EC2 infrastructure and advertises long-term storage using Amazon’s S3 storage service and advanced analytics using Amazon’s Hadoop-based Elastic

MapReduce service. Papertrail aggregates the following.

syslog

Text log files

Apache

MySQL

cloud hosting

Ruby on Rails

Windows events

Tomcat

Heroku apps

Routers & firewalls

Highlights

Instant log visibility: Time-saving log tools,

flexible system groups, team-wide

access, long-term archives, charts and

analytics exports, monitoring web hooks,

and 45-second setup.

Value from collected logs: Track down

customer problems, error messages, app

requests, slow DB queries, config

changes.


Developmental Aids

ApacheBenchApacheBench (ab) is a single-threaded command line computer program for measuring the performance of HTTP web servers. Originally designed to test the

Apache HTTP Server, it is generic enough to test any web server. The ab tool comes bundled with the standard Apache source distribution, and like the

Apache web server itself, is free, open source software and distributed under the terms of the Apache License. It is designed to give you an impression of how

your current Apache installation performs. This especially shows you how many requests per second your Apache installation is capable of serving. Apache-

Bench will only use one operating system thread regardless of the concurrency level (specified by the -c parameter). In some cases, especially when bench-

marking high-capacity servers, a single instance of ApacheBench can itself be a bottleneck. When using ApacheBench on hardware with multiple processor

cores, additional instances of ApacheBench may be used in parallel to more fully saturate the target URL.


Developmental Aids

LogentriesLogentries is a software-as-a-service provider for log management and intelligence. Logentries collects and analyzes data found within log files in real-time

with a cloud-delivered approach. Logentries has more than 25,000 users. The Logentries technology collects and analyzes logs across software stacks using

a pre-processing layer to filter, correlate, and visualize log data. The tool uses a combination of AWS tools and open source applications to deliver the

application.

FeaturesWhether your data is in structured JSON or mysterious plain text, it’s easy to send to Logentries for immediate search.

Organize logs from servers, applications, routers, containers and more in one central location.

Search is really fast, whether one is searching keywords, key value pairs or regex patterns.

Powerful Analytics: Analytical functions such as AVERAGE(), COUNT(), GROUPBY(), min, max and percentile can be used with its intuitive query language,

LEQL

Robust Visualizations: Analyze log data with line graphs, bar charts, multi-line reports and more.

Simple export tools and APIs make it easy to share data outside of Logentries to view it using tools like Tableau, Hosted Graphite or Geckoboard.

Logentries’ unique preprocessing engine enables you to watch your logs stream directly into the Logentries UI with Live Tail and receive immediate alerts

without delay.

Catch unusual activity with live Anomaly Detection based on automated analysis of your systems’ baselines.

Publish alerts to popular tools like Slack, Hipchat, PagerDuty or any webhook for seamless integrations.

REST Alerting API


Developmental Aids

New RelicNew Relic is a software analytics technology delivered in a software-as-a-service (SaaS) model. It monitors Web and mobile applications in real-time that run

in cloud, on-premises, or hybrid environments. The name ‘New Relic’ is an anagram of the name of the founder Lew Cirne. New Relic launched an open SaaS

platform designed to allow developers to deploy 50+ plug-ins from technology partners (or build their own) to the New Relic dashboard. Plug-in technologies

include PaaS/cloud services, caching, database, Web servers and queuing.

The New Relic service has 70 partnerships around the world including include IBM Bluemix, Amazon Web Services, CloudBees, Engine Yard, Heroku, Joyent,

Rackspace Hosting, and Microsoft Azure as well as mobile application backend service providers Appcelerator, Parse, and StackMob. New Relic can be used

to monitor the app’s performance and also other things like database performance, load balancers, caching, or even business data like conversion rates. Any

kind of actionable data can be dreamt up and used for building a plug-in for the New Relic dashboard. New Relic has a fancy API and two SDKs (Ruby and

Java) for programming and a UI builder to keep the dashboard clean and the process fast.

The platform has plug-ins for several technologies like Apache, Nginx, CouchDB and Redis, Varnish and Memcache as well as less popular programs like

SendGrid. The API and SDKs are well maintained and exhaustively documented. Once a plug-in is complete, it can be used privately within your own

organization, or you can join in the free-for-all and publish it publicly to Plug-in Central, a sort of app store for New Relic plugins where others can download

and use your work.


Developmental Aids

Apache SentryApache Sentry is a granular, role-based authorization module for Hadoop. Sentry provides the ability to control and enforce precise levels of privileges on data

for authenticated users and applications on a Hadoop cluster. Sentry currently works out of the box with Apache Hive, Hive Metastore/HCatalog, Apache Solr,

Impala and HDFS. Sentry is designed to be a pluggable authorization engine for Hadoop components. It allows you to define authorization rules to validate a

user or application’s access requests for Hadoop resources. Sentry is highly modular and can support authorization for a wide variety of data models in

Hadoop. Sentry can be used to enforce various access policy rules when accessing data stored on Hadoop Distributed File System through various Hadoop

ecosystem components such as Apache Hive, Apache Pig or others.

Traditionally, user access control in Apache Hadoop has been implemented using file based permissions on HDFS. Following the UNIX permissions model,

HDFS offers all or nothing semantics allowing administrator to configure system to allow certain users or user groups read, write or perform both operations on

files. This system does not enable more fine grained permissions that allow access policies for logical parts within one file. Furthermore, this model can't be

used to restrict access to the rich set of objects in the metadata catalog that are stored outside HDFS.

Sentry was initially developed by Cloudera to allow users fine grained access to data as well as the metadata in Apache Hadoop. Sentry has been maintained

as an open source project on Cloudera’s github. All code in Sentry is open source and is publicly available under the Apache 2 license.


Developmental Aids

Apache JMeterThe Apache JMeter application is open source software, a 100% pure Java application designed to load test functional behavior and measure performance.

Apache JMeter can be used to test performance both on static and dynamic resources (Web Services (SOAP/REST), Web dynamic languages - PHP, Java,

ASP.NET, Files, etc. -, Java Objects, Data Bases and Queries, FTP Servers and more). It can be used to simulate a heavy load on a server, group of servers,

network or object to test its strength or to analyze overall performance under different load types. It can make a graphical analysis of performance or test your

server/script/object behavior under heavy concurrent load.

Apache JMeter features include:

Ability to load and performance test many different server/protocol types:

Web - HTTP, HTTPS

SOAP / REST

FTP

Database via JDBC

LDAP

Message-oriented middleware (MOM) via JMS

Mail - SMTP(S), POP3(S) and IMAP(S)

Native commands or shell scripts

TCP

Complete portability and 100% Java purity.

Careful GUI design allows faster Test Plan building and debugging.


Developmental Aids


Developmental Aids

The Apache JMeter application is open source software, a 100% pure Java application designed to load test functional behavior and measure performance.

Apache JMeter can be used to test performance both on static and dynamic resources (Web Services (SOAP/REST), Web dynamic languages - PHP, Java,

ASP.NET, Files, etc. -, Java Objects, Data Bases and Queries, FTP Servers and more). It can be used to simulate a heavy load on a server, group of servers,

network or object to test its strength or to analyze overall performance under different load types. It can make a graphical analysis of performance or test your

server/script/object behavior under heavy concurrent load.

Apache JMeter features include:

Full multithreading framework allows concurrent sampling by many threads and simultaneous sampling of different functions by separate thread groups.

Caching and offline analysis/replaying of test results.

Highly Extensible core:

Pluggable Samplers allow unlimited testing capabilities.

Several load statistics may be chosen with pluggable timers.

Data analysis and visualization plugins allow great extensibility as well as personalization.

Functions can be used to provide dynamic input to a test or provide data manipulation.

Scriptable Samplers (BeanShell, BSF-compatible languages and JSR223-compatible languages)

Apache JMeter

Amazon Web ServicesAmazon Web Services (AWS), a subsidiary of Amazon.com offers a suite of cloud-computing services that make up an on-demand computing platform. These

include Amazon Elastic Compute Cloud, also known as "EC2", and Amazon Simple Storage Service, also known as "S3". AWS has more than 70 services,

spanning a wide range, including compute, storage, networking, database, analytics, application services, deployment, management, mobile, developer tools

and tools for the Internet of things. Amazon markets AWS as a service to provide large computing capacity quicker and cheaper than a client company

building an actual physical server farm.

Amazon EC2’s simple web service interface allows you to obtain and configure capacity with minimal friction. It provides you with complete control of your

computing resources and lets you run on Amazon’s proven computing environment. Amazon EC2 reduces the time required to obtain and boot new server

instances to minutes, allowing you to quickly scale capacity, both up and down, as your computing requirements change. Amazon EC2 changes the

economics of computing by allowing you to pay only for capacity that you actually use. Amazon EC2 provides developers the tools to build failure resilient

applications and isolate themselves from common failure scenarios.


Developmental Aids

Compute - Amazon Elastic Beanstalk provides a PaaS service for hosting applications, equivalent services include Google App Engine or Heroku or

OpenShift for on-premises use.

Networking - Amazon Virtual Private Cloud (VPC) creates a logically isolated set of AWS resources which can be connected using a VPN connection

Content Delivery: Amazon CloudFront, a content delivery network (CDN) for distributing objects to so-called "edge locations" near the requester.

Storage - Amazon Simple Storage Service (S3) provides Web Service based storage.

Database - Amazon Redshift provides petabyte-scale data warehousing with column-based storage and multi-node compute.

AWS CodeDeploy provides automated code deployment to EC2 instances.

Management - AWS Directory Service a managed service that allows connection to AWS resources with existing on-premises Microsoft Active Directory or

to set up a new, stand-alone directory in the AWS Cloud.

Application Services: Amazon Simple Email Service (Amazon SES) is a cost-effective email service built on the reliable and scalable infrastructure that

Amazon.com developed to serve its own customer base. With Amazon SES, you can send and receive email with no required minimum commitments – you

pay as you go, and you only pay for what you use.

Analytics: Amazon Machine Learning a service that assists developers of all skill levels to use machine learning technology

Amazon Simple Notification Service: Amazon SNS is a fast, flexible, fully managed push notification service that lets you send individual messages or to

fan-out messages to large numbers of recipients. Send push notifications to mobile device users, email recipients or even send messages to other

distributed services. With Amazon SNS, you can send notifications to Apple, Google, Fire OS, and Windows devices, as well as to Android devices in China

with Baidu Cloud Push. You can use SNS to send SMS messages to mobile device users worldwide. Amazon SNS can also deliver messages to Amazon

Simple Queue Service (SQS), AWS Lambda functions, or to any HTTP endpoint.

Products - Examples


Developmental Aids

Amazon Web Services

MailgunMailgun is an online service that provides a set of powerful APIs that allow you to send, receive, track and store email effortlessly. You can control everything

through intuitive, RESTful APIs.

Fast and simple HTTP API or SMTP interface.

Server-side MIME assembly. No libraries required.

Live logs and webhook event notifications.

Everything is built and priced to scale with you.

Easy SMTP integration and a simple, RESTful API abstracts away the messy details of sending transactional or bulk email. Scale quickly, whether you need to

send 10 or 10 million emails.

Route and forward email directly into your app or inbox. Email parsing turns your emails into easy-to-digest structured data and spam filtering keeps out

unwanted emails.

Searchable logs ensures you know what is happening to your email while tags make it easy to A/B test and report on your data, and all via our webhooks.

Advanced email validation to increase conversion. jQuery plug-in enables you to integrate it quickly into your web forms.


Developmental Aids

SendGridSendGrid is a cloud-based transactional email delivery and management service. The service manages various types of email including shipping notifications,

friend requests, sign-up confirmations, and email newsletters. It also handles internet service provider (ISP) monitoring, domain keys, sender policy framework

(SPF), and feedback loops. Additionally, the company provides link tracking, open rate reporting. It also allows companies to track email opens, unsubscribes,

bounces, and spam reports. The company integrated SMS, voice, and push notification abilities to its service through a partnership with Twilio. SendGrid offers

a freemium version and a Lite Plan (pay-as-you-go), as well as three expanded levels of service: Essentials, Pro, and Premier.

SendGrid’s clients include Uber, Twilio, Foursquare, Pandora, and Airbnb.

SendGrid moves transactional mail between web applications and end users. For example, every time content is pinned or re-pinned in Pinterest, an email is

generated to let users know that is happening. Every time something is tagged in a photo, the application generates emails, which goes to the user of that app.

SendGrid sends over six billion emails per month on behalf of those customers. Web developers are good at building web applications. However, every web

application needs an email component. It is easy to set up your own, SMTP server, to send email on behalf of a web app. But one in five emails can go missing,

and the way the Internet hands off emails from web applications to the end user is complicated. SendGrid addresses the problem where an application

generates email to a human, where lots of algorithms are looking for spam and what to block before it gets to the inbox.


Developmental Aids

Campaign TrackerBy adding campaign parameters to your URLs, you can identify the campaigns that send traffic to your site. When a user clicks a referral link, these parameters

are sent to Analytics, so you can see the effectiveness of each campaign in your reports. For example, you could use campaign parameters to find out if more

users arrive at your download page in an app marketplace from an in-app ad or a search ad. You could also use campaign parameters to find out how many

users come to your web site from a link that was included in a monthly email newsletter.

Campaign Tracker supports Google’s campaign variables, which are known as “Source”, “Medium”, “Term”, “Content” and “Name”. When you have campaigns

that use these URL variables the values can be passed directly to your WordPress forms and sent through your forms in a hidden field.

Support for Google Campaign URL variables

Support for addition custom campaign variables (up to six)

Support for leading WordPress form plugins

Support for any number of forms to have tracking configured

Allows you to track and store Google’s GCLID variable (Google Adwords)

Provides referrer information – showing how the visitor arrived at your site

Works with caching plug-in and server side cached environments

New create tracking URLs inside the WordPress editor for each page or post

New store your favorite tracking values inside the plug-in


Developmental Aids

SummaryThe developmental aids discussed here undoubtedly bring about considerable savings in time and effort. There are several more out there. The net result is

that project teams are able to deliver optimum solutions to customers within budget and time to complete satisfaction. If you are interested in knowing more

about third party software for your projects, please contact us. We at SayOne have vast experience of using these for successful deliveries and will be happy to

assist you.


Developmental Aids

www.sayonetech.com

Made with in Kochi

developmental aids whitepaper 9-1-2017 › media › files › white_paper › ... · apache sentry...

Documents