porcelain documentation - read the docs

porcelain DocumentationRelease

<author>

August 20, 2014

Contents

1 Overview 3

2 Installation 5

3 Usage 73.1 Launching one-off programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73.2 Passing input and getting output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73.3 Streams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83.4 Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

4 Configuring the Goon driver 114.1 Setting the driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114.2 Goon options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

5 Going deeper 13

6 Known issues and roadmap 15

7 Acknowledgements 17

8 License 19

9 API Reference 219.1 Porcelain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219.2 Porcelain.Driver.Basic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249.3 Porcelain.Driver.Goon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259.4 Porcelain.Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259.5 Porcelain.Result . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269.6 Porcelain.UsageError . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Elixir Module Index 29

i

porcelain Documentation, Release

Porcelain implements a saner approach to launching and communicating with external OS processes from Elixir. Builton top of Erlang’s ports, it provides richer functionality and simpler API.

Simply put, Porcelain removes the pain of dealing with ports and substitutes it with happiness and peace of mind.

Contents 1


2 Contents

CHAPTER 1

Overview

Having some 20 odd options, the Erlang port API can be unwieldy and cumbersome to use. Porcelain replaces it witha simpler approach and provides defaults for the common cases.

User-level features include:

• sane API

• ability to launch external programs in a synchronous or asynchronous manner

• multiple ways of passing input to the program and getting back its output (including working directly with filesand Elixir streams)

• being able to work with programs that try to read the whole input until EOF before producing output

• ability to send OS signals to external processes (requires goon v2.0)

To read background story on the library’s design and possible future extensions, please refer to the wiki.

3

https://github.com/alco/porcelain/wiki


4 Chapter 1. Overview

CHAPTER 2

Installation

Add Porcelain as a dependency to your Mix project:

def application do[applications: [:porcelain]]

end

defp deps do[{:porcelain, "~> 2.0"}]

end

Now, some of the advanced functionality is provided by the external program called goon. See which particularfeatures it implements in the reference docs here. Go to goon‘s project page to find out how to install it.

5

http://porcelain.readthedocs.org/en/latest/index.html#document-ref/Porcelain.Driver.Goon

https://github.com/alco/goon#goon


6 Chapter 2. Installation

CHAPTER 3

Usage

Examples below show some of the common use cases. See also this demo app. Refer to the API docs to familiarizeyourself with the complete set of provided functions and options.

3.1 Launching one-off programs

If you need to launch an external program, feed it some input and capture its output and maybe also exit status, useexec() or shell():

alias Porcelain.Result

%Result{out: output, status: status} = Porcelain.shell("date")IO.inspect status #=> 0IO.inspect output #=> "Fri Jun 6 14:12:02 EEST 2014\n"

result = Porcelain.shell("date | cut -b 1-3")IO.inspect result.out #=> "Fri\n"

# Use exec() when you want launch a program directly without using a shellFile.write!("input.txt", "lines\nread\nfrom\nfile\n")result = Porcelain.exec("sort", ["input.txt"])IO.inspect result.out #=> "file\nfrom\nlines\nread\n"

3.2 Passing input and getting output

Porcelain gives you many options when it comes to interacting with external processes. It is possible to feed inputfrom a file or a stream, same for output:

File.write!("input.txt", """This file contains some patterns>like this<interspersed with other text... >like this< the end.""")

Porcelain.exec("grep", [">like this<", "-m", "2"],in: {:path, "input.txt"}, out: {:append, "output.txt"})

IO.inspect File.read!("output.txt")#=> ">like this<\n... >like this< the end.\n"

7

https://github.com/alco/porcelain_example

http://porcelain.readthedocs.org


3.3 Streams

Programs can be spawned asynchronously (using spawn() and spawn_shell()) allowing for continuously ex-changing data between Elixir and the external process.

In the next example we will use streams for both input and output.

alias Porcelain.Process, as: Proc

instream = SocketStream.new(’example.com’, 80)opts = [in: instream, out: :stream]proc = %Proc{out: outstream} = Porcelain.spawn("grep", ["div", "-m", "4"], opts)

Enum.into(outstream, IO.stream(:stdio, :line))# div {# div {# <div># </div>

Proc.alive?(proc) #=> false

Alternatively, we could pass the output stream directly to the call to spawn():

opts = [in: SocketStream.new(’example.com’, 80),out: IO.stream(:stderr, :line),

]Porcelain.exec("grep", ["div", "-m", "4"], opts)#=> this will be printed to stderr of the running Elixir process:# div {# div {# <div># </div>

The SocketStream module used above wraps a tcp socket in a stream. Its implementation can be found in thetest/util/socket_stream.exs file.

3.4 Messages

If you prefer to exchange messages with the external process, you can do that:

alias Porcelain.Process, as: Procalias Porcelain.Result

proc = %Proc{pid: pid} =Porcelain.spawn_shell("grep ohai -m 2 --line-buffered",

in: :receive, out: {:send, self()})

Proc.send_input(proc, "ohai proc\n")receive do

{^pid, :data, data} -> IO.inspect data #=> "ohai proc\n"end

Proc.send_input(proc, "this won’t match\n")Proc.send_input(proc, "ohai")Proc.send_input(proc, "\n")receive do

8 Chapter 3. Usage


{^pid, :data, data} -> IO.inspect data #=> "ohai\n"endreceive do

{^pid, :result, %Result{status: status}} -> IO.inspect status #=> 0end

3.4. Messages 9


10 Chapter 3. Usage

CHAPTER 4

Configuring the Goon driver

There are a number of options you can tweak to customize the way goon is used. All of the options described belowshould put into your config.exs file.

4.1 Setting the driver

config :porcelain, :driver, <driver>

This option allows you to set a particular driver to be used at all times.

By default, Porcelain will try to detect the goon executable. If it can find one, it will usePorcelain.Driver.Goon. Otherwise, it will print a warning to stderr and fall back toPorcelain.Driver.Basic.

By setting Porcelain.Driver.Basic above you can force Porcelain to always use the basic driver.

If you set Porcelain.Driver.Goon, Porcelain will always use the Goon driver and will fail to start if the goonexecutable can’t be found.

4.2 Goon options

config :porcelain, :goon_driver_path, <path>

Set an absolute path to the goon executable. If this is not set, Porcelain will search your system’s PATH by default.

11


12 Chapter 4. Configuring the Goon driver

CHAPTER 5

Going deeper

Take a look at the reference docs for the full description of all provided functions and supported options.

13

http://porcelain.readthedocs.org


14 Chapter 5. Going deeper

CHAPTER 6

Known issues and roadmap

• there are known crashes happening when using Porcelain across two nodes

• error handling when using the Goon driver is not completely shaped out

15


16 Chapter 6. Known issues and roadmap

CHAPTER 7

Acknowledgements

Huge thanks to all who have been test-driving the library in production, in particular to

• Josh Adams

• Tim Ruffles

17


18 Chapter 7. Acknowledgements

CHAPTER 8

License

This software is licensed under the MIT license.

19


20 Chapter 8. License

CHAPTER 9

API Reference

9.1 Porcelain

9.1.1 Overview

The main module exposing the public API of Porcelain.

Basic concepts

Functions in this module can either spawn external programs directly (exec/3 and spawn/3) or using a systemshell (shell/2 and spawn_shell/2).

Functions exec/3 and shell/2 are synchronous (or blocking), meaning they don’t return until the external programterminates.

Functions spawn/3 and spawn_shell/2 are non-blocking: they immediately return a Porcelain.Processstruct and use one of the available ways to exchange input and output with the external process asynchronously.

Error handling

Using undefined options, passing invalid values to options or any function arguments will fail with a function clauseerror or Porcelain.UsageError exception. Those are programmer errors and have to be fixed.

Any other kinds of runtime errors are reported by returning an error tuple: {:error, <reason>} where<reason> is a string explaining the error.

9.1.2 Summary

exec/3 Execute a program synchronouslyreinit/1 Reruns the initialization and updates application envshell/2 Execute a shell invocation synchronouslyspawn/3 Spawn an external process and return a Porcelain.Process struct to be able to

communicate with itspawn_shell/2 Spawn a system shell and execute the command in it

21


9.1.3 Functions

exec(prog, args, options \\ [])(function)Specs:

•exec(binary, [binary], Keyword.t/0) :: Porcelain.Result.t

Execute a program synchronously.

Porcelain will look for the program in PATH and launch it directly, passing the args list as command-linearguments to it.

Feeds all input into the program (synchronously or concurrently with reading output; see :async_in optionbelow) and waits for it to terminate. Returns a Porcelain.Result struct containing program’s output andexit status code.

When no options are passed, the following defaults will be used:

[in: "", out: :string, err: nil]

This will run the program with no input and will capture its standard output.

Available options:

•:in – specify the way input will be passed to the program.

Possible values:

–<iodata> – the data is fed into stdin as the sole input for the program

–<stream> – interprets <stream> as a stream of iodata to be fed into the program

–{:path, <string>} – path to a file to be fed into stdin

–{:file, <file>} – <file> is a file descriptor obtained from e.g. File.open; the file will beread from the current position until EOF

•:async_in – can be true or false (default). When enabled, an additional process will be spawnedto feed input to the program concurrently with receiving output.

•:out – specify the way output will be passed back to Elixir.

Possible values:

–nil – discard the output

–:string (default) – the whole output will be accumulated in memory and returned as one string tothe caller

–:iodata – the whole output will be accumulated in memory and returned as iodata to the caller

–{:path, <string>} – the file at path will be created (or truncated) and the output will be writtento it

–{:append, <string>} – the output will be appended to the the file at path (it will be createdfirst if needed)

–{:file, <file>} – <file> is a file descriptor obtained from e.g. File.open; the file will bewritten to starting at the current position

–<coll> – feeds program output (as iodata) into the collectable <coll>. Useful for outputtingdirectly to the console, for example:

22 Chapter 9. API Reference


stream = IO.binstream(:standard_io, :line)exec("echo", ["hello", "world"], out: stream)#=> prints "hello\nworld\n" to stdout

•:err – specify the way stderr will be passed back to Elixir.

Possible values are the same as for :out. In addition, it accepts the atom :out which denotes redirectingstderr to stdout.

Caveat: when using Porcelain.Driver.Basic, the only supported values are nil (stderr will beprinted to the terminal) and :out.

•:dir – takes a path that will be used as the directory in which the program will be launched.

•:env – set additional environment variables for the program. The value should be an enumerable withelements of the shape {<key>, <val>} where <key> is an atom or a binary and <val> is a binaryor false (meaning removing the corresponding variable from the environment). Basically, it accepts anykind of dict, including keyword lists.

reinit(driver \\ nil)(function)Reruns the initialization and updates application env.

This function is useful in the following cases:

1.The currently used driver is Goon and the location of the goon executable has changed.

2.You want to change the driver being used.

shell(cmd, options \\ [])(function)Specs:

•shell(binary, Keyword.t/0) :: Porcelain.Result.t

Execute a shell invocation synchronously.

This function will launch a system shell and pass the invocation to it. This allows using shell features likehaining multiple programs with pipes. The downside is that those advanced features may be unavailable onsome platforms.

It is similar to the exec/3 function in all other respects.

spawn(prog, args, options \\ [])(function)Specs:

•spawn(binary, [binary], Keyword.t/0) :: Porcelain.Process.t

Spawn an external process and return a Porcelain.Process struct to be able to communicate with it.

You have to explicitly stop the process after reading its output or when it is no longer needed.

Use the Porcelain.Process.await/2 function to wait for the process to terminate.

Supports all options defined for exec/3 plus some additional ones:

•in: :receive – input is expected to be sent to the process in chunks using thePorcelain.Process.send_input/2 function.

•:out and :err can choose from a few more values (with the familiar caveat thatPorcelain.Driver.Basic does not support them for :err):

–:stream – the corresponding field of the returned Process struct will contain a stream of iodata.

Note that the underlying port implementation is message based. This means that the external programwill be able to send all of its output to an Elixir process and terminate. The data will be kept in theElixir process’s memory until the stream is consumed.

9.1. Porcelain 23


–{:send, <pid>} – send the output to the process denoted by <pid>. Will send zero or more datamessages and will always send one result message in the end.

The data messages have the following shape:

{<from>, :data, :out | :err, <iodata>}

where <from> will be the same pid as the one contained in the Process struct returned by thisfunction.

The result message has the following shape:

{<from>, :result, %Porcelain.Result{} | nil}

The result will be nil if the :result option that is passed to this function is set to :discard.

Note: if both :out and :err are set up to send to the same pid, only one result message will be sentto that pid in the end.

•:result – specify how the result of the external program should be returned after it has terminated.

This option has a smart default value. If either :out or :err option is set to :string or :iodata,:result will be set to :keep. Otherwise, it will be set to :discard.

Possible values:

–:keep – the result will be kept in memory until requested by callingPorcelain.Process.await/2 or discarded by calling Porcelain.Process.stop/1.

–:discard – discards the result and automatically closes the port after program termination. Usefulin combination with out: :stream and err: :stream.

spawn_shell(cmd, options \\ [])(function)Specs:

•spawn_shell(binary, Keyword.t/0) :: Porcelain.Process.t

Spawn a system shell and execute the command in it.

Works similar to spawn/3.

9.2 Porcelain.Driver.Basic

9.2.1 Overview

Porcelain driver that offers basic functionality for interacting with external programs.

Users are not supposed to call functions in this module directly. Use functions in Porcelain instead.

This driver has two major limitations compared to Porcelain.Driver.Goon:

• the exec function does not work with programs that read all input until EOF before producing any output. Suchprograms will hang since Erlang ports don’t provide any mechanism to indicate the end of input.

If a program is continuously consuming input and producing output, it could work with the spawn function,but you’ll also have to explicitly close the connection with the external program when you’re done with it.

• sending OS signals to external processes is not supported



9.3 Porcelain.Driver.Goon

9.3.1 Overview

Porcelain driver that offers additional features over the basic one.

Users are not supposed to call functions in this module directly. Use functions in Porcelain instead.

This driver will be used by default if it can locate the external program named goon in the executable path. If goonis not found, Porcelain will fall back to the basic driver.

The additional functionality provided by this driver is as follows:

• ability to signal EOF to the external program

• send an OS signal to the program

• (to be implemented) more efficient piping of multiple programs

9.4 Porcelain.Process

9.4.1 Overview

Module for working with external processes launched with Porcelain.spawn/3 orPorcelain.spawn_shell/2.

9.4.2 Summary

__struct__/0A struct representing a wrapped OS processes which provides the ability to exchange data with italive?/1 Check if the process is still runningawait/2 Wait for the external process to terminatesend_input/2Send iodata to the process’s stdinsignal/2 Send an OS signal to the processesstop/1 Stops the process created with Porcelain.spawn/3 or Porcelain.spawn_shell/2.

Also closes the underlying port

9.4.3 Types

tt :: %Porcelain.Process{err: term, out: term, pid: term}

signalsignal :: :int | :kill | non_neg_integer

9.4.4 Functions

__struct__()(function)Specs:

•__struct__ :: %Porcelain.Process{err: term, out: term, pid: term}

A struct representing a wrapped OS processes which provides the ability to exchange data with it.

9.3. Porcelain.Driver.Goon 25


alive?(process)(function)Specs:

•alive?(t) :: true | false

Check if the process is still running.

await(process, timeout \\ :infinity)(function)Specs:

•await(t, non_neg_integer | :infinity) :: {:ok, Porcelain.Result.t} | {:error, :noproc | :timeout}

Wait for the external process to terminate.

Returns Porcelain.Result struct with the process’s exit status and output. Automatically closes the un-derlying port in this case.

If timeout value is specified and the external process fails to terminate before it runs out, atom :timeout isreturned.

send_input(process, data)(function)Specs:

•send_input(t, iodata) :: iodata

Send iodata to the process’s stdin.

End of input is indicated by sending an empty message.

Caveat: when using Porcelain.Driver.Basic, it is not possible to indicate the end of input. You shouldstop the process explicitly using stop/1.

signal(process, sig)(function)Specs:

•signal(t, signal) :: signal

Send an OS signal to the processes.

No further communication with the process is possible after sending it a signal.

stop(process)(function)Specs:

•stop(t) :: true

Stops the process created with Porcelain.spawn/3 or Porcelain.spawn_shell/2. Also closes theunderlying port.

May cause “broken pipe” message to be written to stderr.

9.5 Porcelain.Result

9.5.1 Overview

A struct containing the result of running a program after it has terminated.

9.5.2 Types

tt :: %Porcelain.Result{err: term, out: term, status: term}



9.6 Porcelain.UsageError

9.6.1 Overview

This exception is meant to indicate programmer errors (misuses of the library API) that have to be fixed prior torelease.

9.6.2 Summary

exception/1 Callback implementation of Exception.exception/1message/1 Callback implementation of Exception.message/1

9.6.3 Functions

exception(args)(function)Specs:

•exception(Keyword.t/0) :: Exception.t/0

Callback implementation of Exception.exception/1.

message(exception)(function)Specs:

•message(Exception.t/0) :: String.t/0

Callback implementation of Exception.message/1.

9.6. Porcelain.UsageError 27

Elixir Module Index

pPorcelain, 21Porcelain.Driver.Basic, 24Porcelain.Driver.Goon, 25Porcelain.Process, 25Porcelain.Result, 26Porcelain.UsageError, 27

29

porcelain documentation - read the docs

Documents