D Programming LanguageAn Introduction

Ali Çehreli

February 26, 2015, Axcient Inc.

1

Overview

• High-level system language• Strongly statically typed with type and attribute inference• Fast compiling programs• Fast executing programs• Link compatibility with C and a large subset of C++. (Only

language that can call C++ code (including C++ itself!))• Familiar features from C, C++, Java, C#, Python, Eiffel, and

others• RAII, garbage collection, manual memory management, etc.• Safe memory model• Code safety, program correctness, and pragmatism• Designed mostly by-community

2

Resources

• Main site: http://dlang.org/• Wiki: http://wiki.dlang.org/• Presentations and articles by

◦ Walter Bright: Author of the first C++ compiler (Zortech)and Digital Mars C and C++ compilers; Creator of D1 (2001)and D (2007)

◦ Andrei Alexandrescu: Author of "Modern C++ Design","The D Programming Language" (aka TDPL), and co-author ofothers

◦ DConf speakers (2013, 2014, and upcoming in May 2015)◦ Others

3

Community

• Newsgroups: http://forum.dlang.org/• DConf 2015 will be hosted at Utah Valley University (previous

ones were at Facebook)• Volunteers: https://github.com/D-Programming-Language• Meetups

◦ Bay Area: http://www.meetup.com/D-Lang-Silicon-Valley/◦ Others around the world

4

Success Stories• Don Clugston's DConf 2013 presentation about how Sociomantic

uses DAn excerpt at minute 7:50:http://www.youtube.com/watch?v=pmwKRYrfEyY#t=470"A company entirely based on D, nothing else.""Profitable since day one without any investors at all."(Sociomantic was acquired by UK-based Tesco in 2014, for anundisclosed amount rumored to be between $100-$200 million.)

• Manu Evans's DConf 2013 presentation on how Remedy Games usedDAn excerpt at minute 12:30:http://www.youtube.com/watch?v=FKceA691Wcg#t=765"Turns out that this process is lightning fast thanks to DMD [asopposed to gcc's 20-30 seconds]."

• Facebook for developer tools (see Adam Simpkins's talk at DConf2014)

• D Meetup's sponsor startup Apakau for configuration management• Another startup in SF doing something similar to Sociomantic• Others

5

Example: vibe.dVibed, an asynchronous I/O framework: http://vibed.org/

// Simple HTTP serverimport vibe.d;

shared static this(){

auto settings = new HTTPServerSettings;settings.port = 8080;

listenHTTP(settings, &handleRequest);}

void handleRequest(HTTPServerRequest req,HTTPServerResponse res)

{if (req.path == "/")

res.writeBody("Hello, World!", "text/plain");}

// An echo serverimport vibe.d;

shared static this(){

listenTCP(7, (conn) { conn.write(conn); });}

6

Example: PeggedA Parsing Expression Grammar (PEG) module: https://github.com/PhilippeSigaud/Pegged

import pegged.grammar;

// Note: This is a D string, not D syntax itselfmixin(grammar(`Arithmetic:

Term < Factor (Add / Sub)*Add < "+" FactorSub < "-" FactorFactor < Primary (Mul / Div)*Mul < "*" PrimaryDiv < "/" PrimaryPrimary < Parens / Neg / Pos / Number / VariableParens < "(" Term ")"Neg < "-" PrimaryPos < "+" PrimaryNumber < ~([0-9]+)

Variable <- identifier`));

// Compile-timeenum parseTree1 = Arithmetic("1 + 2 - (3*x-5)*6");

// Run-timeauto parseTree2 = Arithmetic(" 0 + 123 - 456 ");

7

Example: ctRegexThe pattern is parsed at run time:

import std.regex;

auto re = regex (`^.*/([^/]+)/?$`);

Parsed at compile time:

auto re = ctRegex!(`^.*/([^/]+)/?$`);

8

Example: Component ProgrammingWalter Bright's article "Component Programming in D":http://www.drdobbs.com/article/print?articleId=240008321&siteSectionName=architecture-and-design

import std.stdio;import std.array;import std.algorithm;

alias algsort = std.algorithm.sort;

void main() {stdin

.byLine(KeepTerminator.yes)

.map!(line => line.idup)

.array

.algsort

.copy(stdout.lockingTextWriter());}

9

Example: TypeTuplevoid foo(T)(T a, T b){

// ...}

unittest{

import std.typetuple;

foreach (T; TypeTuple!(int, long, double)) {foo(T.init, T.init);

}}

void main(){}

10

Compiled Language• Available compilers:

◦ dmd: Digital Mars D compiler◦ gdc: Digital Mars front-end, GCC back-end◦ ldc: Digital Mars front-end, LLVM back-end◦ sdc: A compiler as a library

• Compile Time Function Execution (CTFE)

enum Menu m = makeMenu([ "Pancake", "Waffle" ]);

m is generated at compile time. (No need for a special keyword.)• Compile-time reflection

static if (__traits(hasMember, T, "copy") &&isSomeFunction!(T.copy)) {

auto tmp = value.copy();// ...

}

static if (__traits(compiles, value * 2)) {auto result = value * 2;

}

11

rdmd for Hash Bang

D programs can be used as shell scripts.

#!/usr/bin/env rdmd

import std.stdio;

void main(){

writeln("Hello world.");}

12

Interfacing with C and C++• Can use C and C++ libraries directly through D bindings

// Sample D bindingmodule my_ncurses;

extern (C):

enum TRUE = 1;enum FALSE = 0;

alias WINDOW = void;

WINDOW * initscr();int cbreak();// ...

extern (C++, outer_namespace.inner_namespace) void foo();

• Yes, it has pointers

int i;int *p = &i;*p = 42;ubyte *u = cast(ubyte*)p;

13

Multi-paradigm• Imperative

auto names = [ "Alice", "Bob" ];

foreach (name; names) {writefln("Hello %s.", name);

}

• Object oriented

class Cat : Animal{

string sing() { return "miao"; }}

• Functional

auto my_lambda = a => a * 2;my_lambda(100);

• Generic (and generative)

T add(T)(T a, T b){

return a + b;}

14

Program Correctness• Philosophy: "The easiest method is the safest method." Examples:

◦ Every variable is initialized by default◦ Data is thread-local by default◦ Array indexing is bounds-checked by default◦ etc.

• SafeD: Code marked as @safe cannot corrupt memory• Integrated unit testing• Contract programming (à la Eiffel's "design by contract") ensures

object invariants and function entry and exit conditions• Exceptions (Destructors can throw; collateral exceptions are

linked to the original one.)

15

Hot off the presses: Sealed ReferencesReturning a reference is useful:

ref int pick(ref int a, ref int b){

return a % 2 ? a : b;}// ...

int a = 1, b = 2;pick(a, b) = 3; // 'a' becomes 3

However it is open to subtle bugs:

ref int foo(int i){

int j = 10;return pick(i, j); // BUG: Returning reference to local data

}

Solution: return parameters meaning "return if lives long enough"

ref int pick(return ref int a, return ref int b){

return a % 2 ? a : b;}

ref int foo(int i){

int j;return pick(i, j); // ← compilation ERROR

}16

SafeD

Functions defined as @safe and modules compiled with -safe cannotcorrupt memory.Examples of what is not allowed in SafeD:• Inline-assembly• Conversions between values and pointers• Potentially unsafe pointer uses

However, T1* can convert to T2* in the safe direction. Forexample, T* → void* or int* → short* .

• Removing const , immutable , or shared attribute• etc.

17

Unit Testing

/*** ... (function documentation) ...*/

string repeat(string s, size_t count){

// ...}

///unittest{

assert(repeat("abc", 2) == "abcabc");assert(repeat("ğ", 5) == "ğğğğğ");assert(repeat("a", 0) == "");

}

Enabled when compiled with the -unittest compiler switch.• /** ... */ defines documentation comments• unittest blocks that are marked with /// are added to the

documentation

18

Contract Programmingin for entry conditions, out for exit conditions:

string repeat(string s, size_t count)in {

assert(!s.empty);}out (result) {

assert(result.length == (s.length * count));}body {

string result;// ...return result;

}

invariant for invariants:

class WallClock{

int hour;int minute;

invariant(){

assert((hour >= 0) && (hour <= 23));assert((minute >= 0) && (minute <= 59));

}// ... member functions that mutate 'hour' and 'minute' ...

}

19

The scope statement

• scope(failure): When the scope is exited due to an exception• scope(success): When the scope is exited normally• scope(exit): When the scope is exited under any condition

The following program either copies the file or does not leave anygarbage behind.

import std.exception;import std.file;

void main(string[] args){

enforce(args.length == 3,"Usage: mycopy <source> <destination>");

auto tmpFile = args[2] ~ ".tmp";scope(failure) if (exists(tmpFile)) remove(tmpFile);copy(args[1], tmpFile);

rename(tmpFile, args[2]);}

Disclaimer: Yes, exists() or remove() can fail as well.

20

Module System

• import , not #include• No #ifndef header guards• No need for declaration and definition separation (but possible)• Definition order does not matter in most cases

module deneme; // The name of this module

import std.stdio; // Using the standard input+output module

void main(){

sayHi(); // No need to forward declare}

void sayHi(){

writeln("Hi!");}

21

Fundamental TypesUnlike C and C++, type widths are standard (except real).

// Boolean type:bool b;

// Unicode encoding types:char c_8;wchar w_16;dchar d_32;

// Integer types:byte b_8;short s_16;int i_32;long l_64;// cent c_128; Reserved keyword

/* Plus the unsigned versions of the integer types:* ubyte, ushort, uint, ulong, ucent */

// Floating point types:float f_32;double d_64;real r_platform_dependent; // e.g. 128 bits

// 'void' as well:void[] bytes;

22

String types are UTF encodings

Unicode support both in source code and in strings:

string déjà_vu = "déjà vu"; // UTF-8wstring déjà_vu_16 = "déjà vu"; // UTF-16dstring déjà_vu_32 = "déjà vu"; // UTF-32

Conveniently iterated by code points:

writefln("%-(%c %)", déjà_vu);

d é j à v u

Equally conveniently iterated by code units:

writefln("%-(%c %)", déjà_vu.byCodeUnit);

d \303 \251 j \303 \240 v u

23

const and immutableconst in C and C++ is not perfect: It is not transitive and data can bemutated by others.• const is similar in D ("I cannot mutate data through this

reference but others may") but it is transitive.• immutable means "immutable" and is transitive as well.

immutable id = 42;string s = "hi"; // string is immutable(char)[]

◦ No need to lock immutable data in concurrency.◦ Compiler has more opportunities to optimize.

Both are transitive:

struct S { int id; }

const module_level_S = S(42);

int foo(ref const(S) s){

s.id = 100; // ← COMPILATION ERRORs = module_level_S; // ← COMPILATION ERROR

}

(It is possible to rebind const class references with Rebindable .)

24

Object Lifetimes

• Garbage collector (GC)◦ Classes by default◦ Dynamic arrays◦ Associative arrays

@nogc attribute ensures that a function does not use the GC.• RAII

◦ Structs by default• Manual memory management

25

struct

D structs are somewhere between C and C++ structs.• Value type• By default, objects are finalized when leaving scopes but they can

be owned by the GC as well.• struct does not support OOP.

struct Point{

double x;double y;

void reflect(){

x = -x;y = -y;

}}

// ...

auto p = Point(10, 20);

Note: Usually, there is no need for a constructor, destructor, or otherspecial functions.

26

class and interface• Reference type• Normally owned by the GC but scoped() can put class objects on the

stack and emplace() on any piece of memory.• Supports OOP with single class-inheritance and multiple interface-

inheritance.

interface Walker{

void walk(int distance);}

interface Singer{

string sing();}

class Cat : Walker, Singer{

void walk(int distance){

foreach (i; 0 .. distance) {writeln("pitter-patter");

}}

string sing(){

return "meow";}

}

27

C Arrays... ?

• Arrays are a convention in C: A pointer to a single variable and alength (or an end pointer) makes an array.

• C arrays are sometimes value types (e.g. when members ofstructs) and sometimes reference types (e.g. when functionparameters)

• C does not have any special array operation (perhaps with theexception of memmove()). For example, elements may have to beadded only after ensuring room explicitly with realloc() .

• Array syntax is inside-out

char region[1][2]; // One char[2] or two char[1]s?printf("%zu", sizeof(region[0])); // Prints 1 or 2?

For more details, see Walter Bright's article "C's Biggest Mistake":http://www.drdobbs.com/article/print?articleId=228701625(Of course, C++ is in much better position with std::vector .)

28

Fixed-length Arrays (aka Static Arrays)

• Fixed number of elements• Value type• Bounds checking can be done at compile time

int[3] array= [ 10, 42, 100 ];assert(array.length == 3);array[0] = 11;int a = array[5]; // ← compilation ERROR

• Syntax is consistent and natural: Type[length]

/* Element types are highlighted: */

char[2] array; // 2 chars

char[2][1] region; // One char[2]writeln(region[0].sizeof); // Prints 2

29

Dynamic Arrays

• Number of elements can vary• Implemented as a pair of pointer and length• ~ operator to concatenate• ~= operator to append

int[] array= [ 10, 42, 100 ];array.length += 10;assert(array.length == 13);

array[0] = 11;array ~= 7; // append

auto newArray = array ~ array; // concatenate

int a = array[20]; // run-time error (by default)

Dynamic arrays are slices...

30

SlicesOne of the most useful features of D. Efficient, convenient, and safe...

int[] array = [ 10, 20, 30, 40 ];int[] slice = array[1..3]; // 20 and 30

Example:

bool is_palindrome(string s){

if (s.length < 2) {return true;

}

return (s[0] == s[$-1]) &&is_palindrome(s[1..$-1]);

}

unittest{

assert(is_palindrome("abccba"));assert(!is_palindrome("abca"));

}

Disclaimer: Correct for ASCII strings only.Skipping the program name in main():

void main(string[] args){

foo(args[1..$]);}

31

Associative Arrays

A hash table implementation.A mapping from string to string:

string[string] colors = [ "red" : "kırmızı","blue" : "mavi","green" : "yeşil" ];

writeln(colors["red"]); // prints "kırmızı"

A mapping from string to double:

double[string] universalConstants;

universalConstants["pi"] = 3.14;universalConstants["e"] = 2.72;

Structs and classes can be key types by overloading the toHash()member function.

32

Functional Programming

• immutable data• pure functions• Delegates and lambdas (enabling closures)• Lazy evaluations are common

import std.stdio;import std.conv;import std.algorithm;

void main(string[] args){

const input = args[1..$];

// Sum of the odd numbers on the command lineconst result = input

.map!(to!long)

.filter!(a => a % 2)

.sum;

writeln(result);}

$ ./deneme 1 2 3 4 59

33

pure Functions

• pure functions cannot access mutable global stateSo, they always produce the same result(s) for a given set ofarguments.

• Different from most other functional languages, pure functions inD can mutate local state, even their arguments!

34

pure Function ExampleMutations are highlighted:

pure long[] fibonacci(size_t n){

long[] result;

if (n > 0) {result ~= 0;--n;

if (n > 0) {result ~= 1;--n;

foreach (i; 0 .. n) {result ~= result[$-1] + result[$-2];

}}

}

return result;}

void main(){

import std.stdio; // Aside: Note local importwriteln(fibonacci(10));

}

[0, 1, 1, 2, 3, 5, 8, 13, 21, 34]

35

Templates• Function templates

auto min(L, R)(L lhs, R rhs){

return rhs < lhs ? rhs : lhs;}

• struct and class templates

struct Point(T = long, int dimensions = 2){

T[dimensions] coordinates;

void reflect(){

foreach (ref coordinate; coordinates) {coordinate = -coordinate;

}}

}

/* Aside: Unlike typeof in C and C++, alias syntax is* natural in D: */

alias Point3D = Point!(double, 3);

Point3D center; // Aside: double.init is double.nan, not 0.

int i = 42;auto s = i.to!string; // No parentheses for single symbol

36

Generic Programming• Templates are powerful and easy

struct MyRange(T)if (isInputRange!T) // template constraint

{// ...

}

• static if

void dumpObject(T)(ref T var){

// ...static if (isArray!T) {

writeln("elements:");// ...

}// ...

}

static if will not be added to C++ (see "“Static If” Considered", document N3613at http://isocpp.org/files/papers/ by Bjarne Stroustrup and others)

• static assert is a part of the language

static assert((T.sizeof % 4) == 0,"The size of the type must be a multiple of 4");

37

Template Mixinsmixin template Edges(T, size_t count){

T[count] edges;

void setEdge(size_t index, T edge){

// ...}

void printEdges(){

// ...}

}

struct Line{

// struct gains a member and two functionsmixin Edges!(int, 2);

}

void main(){

auto line = Line();line.printEdges();

}

38

String mixinsAny string that can be produced at compile-time can be mixed in.

import std.bitmanip;

struct S{

mixin (bitfields!(uint, "x", 2,int, "y", 3,uint, "z", 2,bool, "flag", 1));

}

static assert (S.sizeof == 1); // Four fields in one byte

void main(){

S s;s.flag = true;

}

39

What bitfields mixes in (1/2)pragma(msg, bitfields!(int, "i", 3,

uint, "u", 5));

// With Ali's formatting and comments:struct S{

// Getter for i@property int i() @safe pure nothrow @nogc const {

auto result = (_i_u & 7U) >>0U;if (result >= 4U) result |= 4294967288U;return cast(int) result;

}

// Setter for i@property void i(int v) @safe pure nothrow @nogc {

assert(v >= i_min,"Value is smaller than the minimum value of bitfield 'i'");

assert(v <= i_max,"Value is greater than the maximum value of bitfield 'i'");

_i_u = cast(typeof(_i_u))((_i_u & ~cast(typeof(_i_u))7U) |((cast(typeof(_i_u))v << 0U) & 7U));

}

// Value range for ienum int i_min = cast(int)18446744073709551612UL; // -4enum int i_max = cast(int)3U;

// ... (continued on the next slide) ...}

40

What bitfields mixes in (2/2)struct{

// ... (continuing from the previous slide) ...

// Getter for u@property uint u() @safe pure nothrow @nogc const {

auto result = (_i_u & 248U) >>3U;return cast(uint) result;

}

// Setter for u@property void u(uint v) @safe pure nothrow @nogc {

assert(v >= u_min,"Value is smaller than the minimum value of bitfield 'u'");

assert(v <= u_max,"Value is greater than the maximum value of bitfield 'u'");

_i_u = cast(typeof(_i_u))((_i_u & ~cast(typeof(_i_u))248U) |((cast(typeof(_i_u))v << 3U) & 248U));

}

// Value range for u: [0, 31]enum uint u_min = cast(uint)0U;enum uint u_max = cast(uint)31U;

// The storage for i and uprivate ubyte _i_u;

}

41

Multi-threading Support

• Data is thread-local by default• Only shared data can be shared (__gshared is available as well

for C-style globals)• immutable is implicitly shared• synchronized for easy synchronization• Standard modules

◦ std.parallelism

◦ std.concurrency for message-passing concurrency◦ core.atomic for atomic operations◦ core.thread for Thread , Fiber , and others◦ core.sync.* package for classic synchronization primitives

42

std.parallelism Module

To execute independent operations simultaneously to make theprogram run faster.• Assuming that the following takes 4 seconds on a single core:

auto students =[ Student(1), Student(2), Student(3), Student(4) ];

foreach (student; students) {student.aLengthyOperation();

}

• The following takes 1 second on 4 cores:

foreach (student; students.parallel) {student.aLengthyOperation();

}

43

std.concurrency Moduleimport std.stdio;import std.concurrency;

void main(){

auto worker = spawn(&func);

worker.send(42); // note different types of messagesworker.send("hello");worker.send(Terminate());

}

struct Terminate{}

void func(){

bool done = false;

while (!done) {receive(

(int msg) {// ...

},

(string msg) {// ...

},

(Terminate msg) {done = true;

});}

}

44

Ranges instead of Iterators

• Ranges are for abstracting data structures from algorithms.• They define element access and iteration for a given collection.• See Andrei Alexandrescu's article "On Iteration" for more

information on D's ranges:http://www.informit.com/articles/printerfriendly/1407357

InputRangeempty, front, popFront()

↑

ForwardRangesave()

↗ ↖

BidirectionalRange RandomAccessRange (infinite)back, popBack() opIndex()

↑

RandomAccessRange (finite)opIndex(), length

45

InputRangeCompile-time duck typing of three functions is sufficient to make anInputRange:• empty: Returns true when the range is empty• front: Returns the first element• popFront(): Removes the first element

[ 10, 11, 12 ][ 11, 12 ]

[ 12 ][ ]

struct FibonacciSeries{

int current = 0;int next = 1;

static const empty = false; // ← makes an infinite range

@property int front() const{

return current;}

void popFront(){

int nextNext = current + next;current = next;next = nextNext;

}}

46

InputRange Usage

FibonacciSeries().take(5).cycle.take(20).writeln;

// The same as above (before UFCS)// writeln(take(cycle(take(FibonacciSeries(), 5)), 20));

The output:

[0, 1, 1, 2, 3, 0, 1, 1, 2, 3, 0, 1, 1, 2, 3, 0, 1, 1, 2, 3]

In explicit form:

auto range = FibonacciSeries();auto firstPart = take(range, 5);auto repeated = cycle(firstPart);auto firstPartOfRepeated = take(repeated, 20);

writeln(firstPartOfRepeated);

Evaluated lazily. Actual operations are inside popFront() as theranges are consumed (by writeln in this example).

47

A Range Using Another Rangestruct Negate(R)

if (isInputRange!R){

R range;

@property bool empty() { return range.empty; }

@property auto front() { return -range.front; }

void popFront() { range.popFront(); }

static if (isForwardRange!R){

Negate!R save() { return Negate!R(range.save()); }}

}

Negate!R negate(R)(R range){

return Negate!R(range);}

writeln(FibonacciSeries().take(5).negate);

The output:

[0, -1, -1, -2, -3]

48

Moving rvalues in C++struct S {

vector<int> v_;

S(const vector<int> & v) // construct with lvalue: v_{v} {}

S(vector<int> && v) // construct with rvalue: v_{std::move(v)} {}

S(const S & that) // copy from lvalue: v_{that.v_} {}

S(S && that) // move from rvalue: v_{std::move(that.v_)} {}

// WARNING: v_{that.v_} would still copy!};

vector<int> make_vector() { // returns rvaluevector<int> v;// ...return v;

}

S make_S() { // returns rvaluereturn S(/* ... */);

}

int main() {vector<int> v{ 1, 2, 3 };

S s0{v}; // copies vS s1{make_vector()}; // moves rvalueS s2{s0}; // copies s0S s3{make_S()}; // moves rvalue

}

49

No class rvalues in D

Not an issue:

class C{}

C make_C() // returns class reference, not rvalue{

return new C();}

void main(){

auto c = make_C();}

50

Moving struct rvalues in Dstruct S { // ← Nothing special needed; already efficient!

Vector v;}

Vector make_vector() { // returns rvalueVector v;// ...return v;

}

S make_S() { // returns rvaluereturn S();

}

void main() {auto v = Vector([ 1, 2, 3 ]);

auto s0 = S(v); // copies vauto s1 = S(make_vector()); // moves rvalueauto s2 = s0; // copies s0auto s3 = make_S(); // moves rvalue

}

struct Vector {int[] elements;

this(this) { // ← post-blitelements = elements.dup;

}}

51

Moving struct rvalues in D - How:Moving rvalues is handled by the language automatically and safely.(Note: D forbids structs having internal-pointers.)

Lvalue on the right-hand sidedst = src;

// Equivalent pseudo-code{

// Make an actual copy of src (maybe expensive and may throw)auto tmp deep-copy src;

dst bit-swap tmp;

} // 'tmp' destroys the old state here

Rvalue on the right-hand sidedst = make_S();

// Equivalent pseudo-code{

dst bit-swap rvalue;

} // 'rvalue' destroys the old state here

52

User-defined Attributes (UDA)struct Obfuscated{

// ...}

struct User{

string name;@Obfuscated string password;

}

void serialize(T)(T object){

foreach (member; __traits(allMembers, T)) {// ...static if (hasAttribute!(fullName, Obfuscated)) {

// ... obfuscate this member ...}// ...

}}

void main(){

auto user = User("Alice", "mypetsname");serialize(user);

}

name: Alicepassword: nzqfutobnf

53

The End

54