filesystem abstraction with flysystem

FILESYSTEMABSTRACTION

WITH FLYSYSTEMBy @FrankDeJonge

WHAT IS FILESYSTEM ABSTRACTION?Bridging the gap between filesystems so the storage engine

becomes an implementation detail.

FILESYSTEMABSTRACTION 101

What are filesystems?How do we use filesystems?Moving to remote filesystems.Abstraction saves the day.

WHAT IS A FILESYSTEM?

LOGIC & STRUCTURE

WHAT IS A FILESYSTEM?Logic to structure and organize pieces of data in a storage

system.

WHAT'S IN A FILESYSTEM?FilesDirectoriesMetadata

COMMON METADATALocation / PathTimestampsPermissions / Ownership

FILE SPECIFICMime-typeFile sizeContents

DERIVED INFORMATION / PATHINFO():FilenameBasenameExtensionDirnameMime-type

FILESYSTEM STRUCTURE

STRUCTURE TYPES:Nested or Linear

src/

Filesystem.php

AdapterInterface.php

tests/

Filesystem.php

AdapterInterface.php

src/Filesystem.php

src/AdapterInterface.php

tests/Filesystem.php

tests/AdapterInterface.php

NESTED FILESYSTEMSAre the most commonHave directorieswhich have files and more directories... which have more files and directories... which have more files and directories.We're really used to this structure.

LINEAR FILESYSTEMAre like key/value storesdo have files, of courserequire a different approachdon't really have directories.

DIRECTORIES ARE A MYTH

WHAT CAN WE DO WITHA FILESYSTEM?

Several operations:

writereadupdatedelete

move / renamecopylistinspect

HANDLING FILESYSTEMS WITH PHP, A WORLDOF HURT.

In vanilla PHP for local files:

$result = file_put_contents($location, $contents);

$contents = file_get_contents($location);

unlink($location);

$timestamp = mtime($location);

$mimetype = mime_content_type($location); // DEPRECATED

$mimetype = (new Finfo(FILEINFO_MIME_TYPE))->file($location);

LISTING FILESglob($location . '/*'); // ?

scandir($location); // ?

readdir(opendir($location)); // ?

array_map('normalize_fileinfo',

iterator_to_array(new DirectoryIterator($location))); // ?

array_map('normalize_fileinfo',

iterator_to_array(new RecursiveIteratorIterator(

new RecursiveDirectoryIterator($location)

))); // ?

Memory problems? Use streams:

$resource = fopen($rLocation, 'r+');

$handle = fopen($location, 'w+');

while ( ! feof($resource)) {

fwrite($handle, fread($resource, 1024), 1024);

}

$result = fclose($handle);

fclose($resource);

WORKING WITH OTHERFILESYSTEMS.

BECAUSE REASONS.

SOME REASONS WHY:We want to scale (horizontally).... so, we want stateless apps.We don't have to serve files ourselves.Share files across multiple application.We want to enable a nice experience to our users which wasnot possible otherwise.

SOME CONSEQUENCESWe can't use built-in PHP function anymore.We'll have to start using API'sWe'll have to depend on third party code.

COMPARING INTEGRATIONS: WRITING FILESAWS/S3:

$options = [

'Body' => $contents,

'ContentType' => 'plain/text',

'ContentLength' => mb_strlen($contents),

];

$result = $awsClient->putObject($options);

Dropbox:

$result = $dropboxClient->uploadFileFromString(

$location,

$contents,

WriteMode::add());

Rackspace:

$result = $rackspaceClient->uploadObject($location, $contents);

WebDAV:

$result = $client->request('PUT', $location, $contents);

FTP:

$stream = tmpfile();

fwrite($stream, $contents);

ftp_fput($connection, $destination, $stream, FTP_BINARY);

fclose($stream);

DOWNSIDESDifferent APISupport different kinds of input (string vs stream support)Different results for EVERY operation

CHOOSING A FILESYSTEM ...can create technical deptcan create vendor lock-inrequire you to invest in them

SOLUTION?

USE AN ABSTRACTFILESYSTEM.

WHAT IS AN ABSTRACT FILESYSTEM?Logic and structure that enable a generalized API to work with

different filesystems.

WHAT DOES A ABSTRACT FILESYSTEM DO?It bridges the gap between different filesystem

implementations/packages and normalizes responses.

INTRODUCING:

FLYSYSTEMMANY FILESYSTEMS, ONE API

FLYSYSTEM IS:A filesystem abstraction layerLike a DBAL for files.Like illuminate/filesystem on steroids.

PACKED WITH SUPERPOWERS:Well tested codeEasy APICachable metadataEasy stream handling

FIRST LOOK AT FLYSYSTEMuse League\Flysystem\Adapter\Local;

$adapter = new Local(__DIR__.'/path/to/dir');

$fs = new Filesystem($adapter);

BASIC FS OPERATIONS:$fs->write($location, $contents);

$fs->read($location);

$fs->update($location, $contents);

$fs->listContents($location, $recursive);

$fs->delete($location);

$fs->getTimestamp($location);

USING AWS S3use League\Flysystem\Adapter\AwsS3;

$adapter = new AwsS3($client, 'bucket-name');

$fs = new Filesystem($adapter);

BASIC FS OPERATIONS:$fs->write($location, $contents);

$fs->read($location);

$fs->update($location, $contents);

$fs->listContents($location, $recursive);

$fs->delete($location);

$fs->getTimestamp($location);

USING FTPuse League\Flysystem\Adapter\Ftp;

$adapter = new Ftp($ftpSettings);

$fs = new Filesystem($adapter);

BASIC FS OPERATIONS:$fs->write($location, $contents);

$fs->read($location);

$fs->update($location, $contents);

$fs->listContents($location, $recursive);

$fs->delete($location);

$fs->getTimestamp($location);

MANY FILESYSTEMS

ONE API

FLYSYSTEM AND STREAMS:$resource = $dropbox->readStream($location);

$awsS3->writeStream($destination, $resource);

FLYSYSTEM IS A GATEWAY DRUG FOR...... DRY code... centralized problem domain handling... easily testable FS interactions... reduced technical dept... lower development costs... having less to learn.

TESTING FILESYSTEM INTERACTIONSUSING FLYSYSTEM

Becomes easier.Becomes more reliable.Might even become fun again.

DEPENDING ON FLYSYSTEMclass Author

{

public function __construct(FilesystemInterface $filesystem)

{

$this->filesystem = $filesystem;

}

public function getBio()

{

return $this->filesystem->read($this->getBioLocation());

}

}

TESTING THE DEPENDANT CODEpublic function testGetBio()

{

$expected = 'Some Bio';

$fsMock = Mockery::mock('League\Flysystem\FilesystemInterface');

$fsMock->shouldReceive('read')

->with('james-doe-bio.txt')

->andReturn($expected);

$autor = new Author($fsMock);

$this->assertEquals($expected, $author->getBio());

}

BONUS: this makes tests run super fast because you don't haveto wait for blocking filesystem operations (slow).

IMPROVED STABILITY OF DEPENDANT CODEClass should have one reason to change... changing filesystems isn't one of them.Most code shouldn't have to care where files are stored.Flysystem enables classes to be unaware of this.

OPENING UP TO NEW POSSIBILITIESUsing Dropbox for uploadsEase development by deferring filesystem choice.Painless pivots during application lifecycle.Nice to build a package on.

FLYSYSTEM INTEGRATIONSLaravel Integration:https://github.com/GrahamCampbell/Laravel-FlysystemSymfony Bundle:https://github.com/1up-lab/OneupFlysystemBundleBackup Manager:https://github.com/heybigname/backup-managerCartalyst Media:http://cartalyst.com

CRAZY FLYSYSTEM ADAPTERSNOTE: some might be urban myths, I haven't seen them all.

FlickrYoutubeReddit

THE LEAGUE OF EXTRAORDINARY PACKAGES

HTTP://GITHUB.COM/THEPHPLEAGUE/FLYSYSTEM

THANKS FOR LISTENING!Find me on twitter @FrankDeJonge