slideall

Copyright © 2005,2006, Center of the International Cooperation for Computerization (CICC) All Rights Reserved.Copyright © 2005,2006, Mitsubishi Research Institute, Inc. All Rights Reserved.Copyright © 2008, University of Puerto Rico at Mayaguez. All Rights Reserved.

1An Introduction to Free/Open-Source Software

(1) What is Free/Open-Source Software(FOSS)?

Why choose Free/Open-Source Software?The processes involved in writing software

understand the importance of source code to software

FOSS and proprietary softwareDifferences between FOSS and proprietary software

Releasing source codethe merits of releasing source code

Misunderstandings and the truths about FOSSFOSS engineers and the career paths

examine the skills required for FOSS engineers, and the career paths



Why Choose Free/Open-Source Software?

Reasons for interest in FOSS

Standards (open standards)Value, quality, innovationFreedom of choiceA lot of flexibilitySecurityCost

Purpose of this courseLearn about significance of FOSSUnderstand benefits of FOSSLearn how to use FOSS

ContentsWhat is FOSS?Major FOSS Packages and DistributionsPhilosophy and History of FOSSFOSS Development and FOSS CommunityFOSS in Business and Case StudiesFOSS and Government Policy, E-governmentFeatures and Issues with FOSSDevelopment ToolsSoftware Components and Examples of Application Building



Process of Writing Software

Operating principles of computersConfirming the basics of computers

Binary code and source codeWhy the source code is so important?

Compilers and interpretersDivided into two categories

Programming languagesOverview of typical programming languages



Operating Principles of Computers

Von Neumann architectureExecutes a sequence of instructions stored in memoryMachine language

Binary codeDifficult for people to readEvolution from 8-bit to 16-bit, then 32-bit and now 64-bit

Computer configurationCentral processing unit (CPU)Storage devices

Primary storage device (memory)Secondary storage device (external memory, hard drive, etc.)

Input/output devicesLCD/CRT display, keyboard, mouse, printer, etc.



Binary Code and Source Code

Binary codeMachine language (native code)

Example of instruction set directly executable by CPURepresented by hexadecimal numbers

Byte-codeExecuted by virtual machineUsed for Java, etc.

Source codeProgramming language

Understandable to peopleModifiable

Requires conversion to binary codeConversion by compiler, byte-code compiler or interpreter

0010000 0a29 2020 2020 2023 6874 7369 6920 20730010020 2061 6574 706d 726f 7261 2079 6168 6b630010040 7520 746e 6c69 7720 2065 6163 206e 696c0010060 6576 7720 7469 2068 6874 2065 6564 61660010100 6c75 2074 6573 7261 6863 7020 7461 73680010120 200a 2020 6920 2066 205b 4c24 5f44 494c0010140 5242 5241 4e59 3233 505f 5441 2048 3b5d0010160 7420 6568 0a6e 2020 2020 2020 5320 53590010200 4554 5f4d 444c 4c5f 4249 4152 5952 334e0010220 5f32 4150 4854 243d 444c 4c5f 4249 41520010240 5952 334e 5f32 4150 4854 200a 2020 20200010260 2020 7865 6f70 7472 5320 5359 4554 5f4d0010300 444c 4c5f 4249 4152 5952 334e 5f32 41500010320 4854 200a 2020 2020 2020 444c 4c5f 42490010340 4152 5952 334e 5f32 4150 4854 3a3d 24220010360 6473 705f 6f72 2267 243a 444c 4c5f 4249

/* * Initialize directory-related fields in the mount structure. */static voidxfs_dir_mount(xfs_mount_t *mp){

uint shortcount, leafcount, count;

mp->m_dirversion = 1;shortcount = (mp->m_attroffset -

(uint)sizeof(xfs_dir_sf_hdr_t)) / (uint)sizeof(xfs_dir_sf_entry_t);

leafcount = (XFS_LBSIZE(mp) -(uint)sizeof(xfs_dir_leaf_hdr_t)) / ((uint)sizeof(xfs_dir_leaf_entry_t) + (uint)sizeof(xfs_dir_leaf_name_t));



Compilers and Interpreters

CompilerConverts source code to binary code during compilingAdvantages of compiled languages

Low overhead during execution; high-speed executionDrawback of compiled languages

Changes in source code require recompiling

Interpreter (scripting languages are also a type of interpreted language)

Source code interpreted at each executionAdvantages of interpreted languages

No compiling required; easy to create codesDrawbacks of interpreted languages

Inferior performance during execution; not suited for large-scale systems

Some interpreted languages are first compiled each time to an intermediate language before being executed



Programming Languages

Major compiled languagesC language

Object-oriented extensions of CC++Objective-C

Conversion: C -> Assembler* -> Machine language*Assembly language corresponds nearly code-for-code with machine language, but is designed for people to read

FORTRAN, PascalProgramming languages that use byte-code interpreter (Virtual Machine type)

Java, C# (.Net)

Major interpreted languagesPerl, PHP, Python, BASIC, LISP, Ruby and many others



FOSS and Proprietary Software

Software as a productDifference between software product and 'goods'

Issues with software productsProblems such as illegal copy and piracy

CountermeasuresWhat were the countermeasures against such issues in the past?

Emergence of Free/Open-SourceSpotlighted as a novel software paradigmTurning point of software business



Software as a Product

Traditionally, software had a strong “freebie” element

Accessories to hardwareOS, applications and other minute softwareBasic reservations about paying for immaterial goods

Commoditization of computers (from mid-90s)Package software became commonplace

Proprietary software emerged as product

Difference between software and material goodsNegligible cost to copy

Rampant illegal copying and piracy

Difference between software and information content (music, film)

Software requires 100% integrity (zero tolerance for “noise” or data corruption)



Issues with Software Products

Illegal copying, piracyCasual copyingParticularly rampant in Asia (but incidence is not necessarily zero in US/Europe)Accelerated by developments in P2P technology

WinnyWinMXgnutella

Demise of copyright system?

VulnerabilitiesSecurity holesBugsIncreased complexity of software



Countermeasures

Measures against illegal copying

Hardware-based solutionsUSB dongles, etc.

Copy protection technologySuperdistribution systemsDigital watermarks

Software-based countermeasures

License keysSerial keys

Non-technology solutionsComprehensive licensing agreementsEducational initiatives

Measures to address vulnerabilities

PatchesSecurity patchesBug patches

Automatic updatesCost of maintenance cannot be ignored



Emergence of FOSS

Freedom to copySolution to piracy issue

Does not equate to discarding copyright

Software use managed through license

Can counter the risk of vulnerabilitiesFixes implemented by worldwide developer baseIf you have the expertise, you can fix it yourself

Paradigm shift in software productsShift toward being paid for services

Support servicesCustomizationProvide total solutions



Releasing Source Code

Levels of publishing source codeInadequate just releasing source code

Effects of publishing source codeMany benefits for both users and developers



Levels of Releasing Source Code

1.Readable source codeUseful for technology acquisition and security auditEx. Early Unix, Shared Source (Microsoft)

2.Modifiable source code for local useFor customization or tuningEmbedded use is permittedSource code may not be redistributed

3.Freely usable, modifiable and redistributable source code

Qualifies as FOSSEnables smooth implementation of bazaar-style joint developmentDistributable as part of a distribution



Benefits of Releasing Source Code

Benefits for developers and development projects

Can turn to others for help (someone is likely)to debug the softwareto add new featuresmodify the source code out of needto handle maintenance

Benefits for usersFor users capable of modifying source code

Ability to fix by oneself if problem occurs (maybe)Ability to perform detailed customization (maybe)Learn operating principles and acquire technology

Other usersIndirectly benefit from releasing of source code



Truth and Misunderstanding about FOSS

“Open Source” is a proper nounThat is also trademarked

“Free Software” and “Free Beer”Freedom and free-of-charge

Is it adequate only publishing source code?More than just releasing source code

FOSS movement is neither totalitarian nor

communistBased on selfish idea rather than altruistic

cf. “Benefits of Releasing Source Code”



“Open Source” is a Proper Noun

Terms that are proper nouns by definition:“Free Software”“Open Source Software”

“Open Source” is trademarked in the US by Open Source Initiative

Becoming common nounsOther examples of common nouns from proper nouns

Walkman (portable stereo)Rolodex (rotary file)Xerox (copier machine)

Leads to misconceptions, controversy, false rumors and lies

Needs to be properly understood



“Free Software” and “Free Beer”

“Free Software” is freedom softwareFree Software is frequently distributed without charge, but does not have to be free of charge

You can still conduct business with Free Software

Confusion over the meaning of Free SoftwareRichard M. Stallman admits to poor choice of wordsStallman prefers “freedom software”

Freedom of software is not assured under e.g. freeware and shareware

“Free Beer” is beer without chargeFreedom of beer is not guaranteed



More Than Just Publishing Source Code

Publishing source code doesn’t make it Free/Open-Source

One of the major misconceptions about FOSSMust meet Free/Open-Source Software criteria

Critical to release source code under license consistent with OSD

Essence of copyleft not about releasing source code

FOSS checklistDoes released source code actually run?Is it compilable?Modifying source code permitted?Redistribution of modifications permitted?



FOSS Technology and Career Paths

Skill set for FOSS engineersWhat skills are required for FOSS engineers?

Types of FOSS engineersWhen and what FOSS technologies are used?

Skill Matrix for FOSS engineersParticular requirement for each types of FOSS engineers

Career Paths for FOSS engineersWhat career paths are provided to engineers who knows FOSS technologies?



Skill Sets for FOSS Engineers

FOSS fundamentalsUnderstanding FOSS (01)Getting involved in FOSS development (02)

Unix systemsUnix operation (l1)Unix system management (l2)Unix server management (l3)

FOSS development environment

FOSS development tools (D1)FOSS software components (D2)

Development basicsComputing basics (C1)Computer languages (C2)System development techniques (C3)Project management (C4)

TechnologyChoose as necessary

Databases (T1)Networks (T2)Web services (T3)Middle-ware (T4)Multimedia (T5) etc...



Types of FOSS Engineers

FOSS system engineersDevelop, maintain and manage systems that use FOSS

System administratorsSystem integrators

FOSS application engineersApplication developers

Use FOSS tools, languages or components to develop applications (contract development)

Package developersUse FOSS tools, languages or components to develop package software

FOSS developersEngineers who develop the FOSS itself



Skill Matrix for FOSS Engineers

* Legend - To be learned basic skills - To be learned advanced skills - Central skills for the engineer

* Skill level - Level 1 : Junior engineers who needs others help - Level 2 : Senior engineers that can do their jobs independently - Level 3 : Leading engineers in their section

Training course for Linux

certification

SecuritySeminar

We need this typesof training course!!

PBL(Project based

Learning)

Classroom lecturesHands-on trainings

Code r

eadin

g

Netw

ork

inte

gra

tion

FOSS development

Skill Level 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3

O1 Understanding OSS

O2 OSS Community Participation ¿

I1 UNIX Operation

I2 UNIX System Administration ¿

I3 UNIX Server Administration ¿

D1 OSS Software Components ¿

D2 OSS Development Tools ¿ ¿

Common

C1 Basics of Computer

C2 Computer Language ¿ ¿ ¿

C3 System Development Method ¿ ¿ ¿

C4 Project Management ¿ ¿ ¿

Technology

T1 Database ↑ ↑ ↑ ↑ ↑ ↑

T2 Network ¿ ¿ ¿ ¿

T3 Web Service ¿ ¿ ¿ ¿ ¿ ¿

T4 ↓ ↓

T5 Multimedia ¿ ¿ ¿ ¿

T6 Platform ¿ ¿ ¿ ¿

T7 High Performance ↓ ↓ ↓ ↓

T8 Security ¿

T9 Standardization

T10 Legacy Migration

System Administrator

System Integrator

Application

Developer

Package Developer

OSS Developmen

t

OSS

Common

UNIX

System

Integra

tion

OSS

Dev.

Env.

Middleware

PMBOKSeminar

We need this typesof training course!!

FO

SS

tun

ing

Un

ivers

ity

/ P

rofe

ssio

nal

sch

ool

Soft

ware

Deve

lop

men

t T

rain

ing

Cours

e

Web

sys

tem

deve

lop

men

t

Pack

ag

e deve

lopm

en

t



Career Paths for FOSS Engineers

SystemAdmin-istrator

SytemIntegrator

ProjectManager

ITArchtect

Con-sultant

FOSS Specialist

FOSS specialist can offer technicaladvice, on FOSS selection, FOSS con-figuration for effective use, etc.(He/she plays a role as FOSS somme-lier for their companies and participatesin FOSS communities outside)

The scope of FOSS skill set

Mar

ketin

g

Consu

ltant

Proje

ct

man

agem

ent

IT S

pecia

list

Softw

are

devel

epm

ent

Operat

ion

Sales

IT A

rchite

ct

Applicat

ion

spec

ialis

t

Custom

er

serv

ice

Educa

tion

Lvl.7

Lvl.6

Lvl.5

Lvl.4

Lvl.3

Lvl.2

Lvl.1

High le

vel

Mid

dle le

vel

Entr

y le

vel

Package/FOSS

develop-ment

ApplicationDevelop-

ment

IT Specialist



(2) FOSS Application Fields and Installation of FOSS

FOSS application fieldsAn introduction to FOSS application fields

Deploying Free/Open-Source SoftwareHow to try FOSS applications

Installing GNU/LinuxExamining GNU/Linux installation process



FOSS Application Fields

Network serversnetwork servers were the quickest to adopt FOSS

Internet business and enterprise systemsbecame popular from B2B to enterprise systems

Embedded systemsFOSS is widely used in embedded environment

EWS (Engineering Work Stations)descend from Unix workstations

Desktopsusability needs to satisfy average users



Network Servers

Very high affinity between FOSS and network services

Evident from origins of FOSS

Internet Service Providers (ISPs)FOSS used for mail servers and name serversPortal sites, Web servers

Load balancer also critical for large-scale ISPs

Small-scale network serversIn-house intranet systems

Mail servers, file serversGroupwareWiki for information sharing



Internet and Enterprise Systems

Evolved from network servers

Factors driving FOSS use for Internet businessFOSS servers used as platformsEmerging IT companies and Internet venture firms

Desire to maximize return by reducing system investment costs

Popularity of e-commerce

Making inroads into enterprise sectorFOSS gradually making inroads into business applications

E-learning, ERP, CRMBusiness package software for specific applications



Embedded Systems

FOSS penetration in embedded environmentsGNU/Linux, NetBSD and other FOSS operating systems support many different CPU architecturesVarious embedded platforms are supported

Motivation for porting often evolves out of developer interest

Embedded equipment manufacturers look to FOSSEMBLIX (since 2000), CE Linux Forum (since 2003)Main equipment applications

Portable information devicesPDAs, mobile phones, car navigation systems

Information appliancesHard disk recorders, media servers, multimedia equipment

Various home electric appliances, as their features become more sophisticated

Refrigerators, air-conditioners, microwave ovens, etc.



Engineering Workstations (EWS)

From Unix to PC-Unix (GNU/Linux, *BSD)Same system operation and user interfaceSame applications are used

Or many comparable FOSS can be used

Can reduce hardware costsSwitching from Unix machines to IBM PC-compatible machines leads to significant cost savings

FOSS development supported by EWS usersContinue to uphold the principle of “user as developer”



Desktops

Desktop use by general users

Last stronghold of proprietary software?Many users only familiar with Windows or Mac OSEase of use nearly the same for all desktop environments

Application fieldsUsed in schools

Used by teaching staff in their officesUsed by children in PC labs

Used in routine task applicationsTelephone operator terminals, counter terminalsMedical data terminals

Office terminals (private sector and government)



Deploying FOSS

Deploying a new FOSS environmentDual bootingCD bootingUsing a Virtual Machine

Using FOSS under WindowsThe first step is trying to use FOSS applications on Windows

CygwinThe package to use GNU software on Windows



Dual Booting

Install multiple operating systems on one systemSwitch between OS’es at bootupPossible to share data by setting up shared drive partition accessible to multiple OS’es

AdvantagesOperation is same as single boot environmentRuns on one machine; affordable way to try new OS

DrawbacksMultiple OS’es cannot be used concurrentlySlightly bothersome to partition hard drive and configure boot loader



CD Booting

OS boots directly from CD-ROMLeading example: Knoppix

Can run on diskless systemUser data stored in USB memory, etc.Some OS’es save data to CD-R at shutdownTemporary files operate on RAM disk

AdvantagesEasy to try new environmentCD-ROM based, so minimal risk of harming system

DisadvantagesSystem cannot be extended

Cannot apply security patches, etc.



Using a Virtual Machine

Run one OS on top of another

Ex: Windows on GNU/Linux

Leading examplesVMwarecoLinux

AdvantagesEasy to try new OS

DrawbacksSlower performance due to inevitable overhead (< 10%)

GNU/Linux running inside of GNU/Linux



Using FOSS on Windows

Growth of FOSS in tandem with Unix development

Linux and GNU softwareX Window System and related softwareUnix server software, etc.

FOSS and running on Unix are fundamentally unrelated

First step in deploying FOSSTry FOSS designed to run on WindowsMany FOSS programs also run on Windows

Apache, PostgreSQL, MySQL, Perl, etc.FOSS written in Java

“Write Once, Run Anywhere”

Try Cygwin



Cygwin

Implements GNU/Linux-like environment on Windows

Two major componentsAPI (cygwin1.dll) for emulating Unix APIsGNU development tool-chain

Excellent portabilityFOSS for Unix will (often) run on Windows, if compiled from source code

X Window System also ported to Windows on Cygwin

Similar software:Services for Unix (SFU) from MicrosoftSFU is free of charge, but not FOSS



Installing GNU/Linux

Example: Installing Fedora Core distribution

Process of installation1. Running the installer2. Basic configuration3. Drive formatting / partitioning4. Network configuration5. Time zone selection and root password setting6. Package installation7. Configuration after software installation (date, display,

and other settings)



Running the Installer

Popular methodGUI installerBased on X Window System; supports mouse operation

If GUI cannot be usedNot supporting GUI

Due to special display

Use classic CUI installer

Installer startupBoot from CDBoot from networkBoot from disk image on hard drive, etc.



Basic Configuration

Language selectionChoose main language to use with systemKeyboard configuration

Many keyboard configurations, depending on the language

Choose installation typeDefault options

Desktop, workstation, server

Custom installation



Drive Formatting/Partitioning

Select hard drive to install toFormatting and partitioningOptionally use installer’s default settings

Formatting and partitioning toolsfdisk (classic tool)Disk Druid

Also configure boot loader at this point



Network Configuration

Configure network environmentHandling of IP addresses

Startup using DHCPAssign fixed IP address

Hostname

Decide network security configurationConfigure firewallEnable/disable remote loginEnable/disable SELinux



Time Zone Selection, Root Password Setting

Time zone selectionPuerto Rican users should select America/Puerto_RicoOr select time zone by using mouse to click on world map

Root password settingPassword for root accountGeneral user accounts can be added later



Package Installation

Choosing software packages to installInstallation type determines which packages are installedIf you chose Custom installation, choose each software package to install

Software packages are sorted by groupDesktopsApplicationsServersDevelopment, etc.



Package Installation (Cont’d)

Installation of software packagesHard drive formatting also performed during this stepMost time-consuming part of installation process

If distribution spans several CD-ROMs, you will be required to change CDs during installation

Software packages can also be added later onRestart computer after software installation finishes



Configuration After Software Installation

Detailed configuration of individual softwareSeparately configure installed softwareDisplay configuration is important

Agree to License AgreementFedora Core asks users to agree to license during this step



Date, Display and Other Settings

Setting the dateUsually set by default to hardware clock

Display settingsConfiguring X Window System

Usually use default settings

Other settingsConfigure sound card, add general user accounts, etc.



(3) Major FOSS and Distibutions

OS and middlewareOS (Operating System) : the basis of systemsMiddleware: software parts between applications and OS

ServersFOSS became popular initially as server applications

DesktopsFOSS applications are expected to be suited for desktops

Development environmentCost merits for starting development easily

Major GNU/Linux distributionsProviding an overview of GNU/Linux distributions



OS (Operating System)

GNU/LinuxThe leading free/open-source OS

FreeBSD/NetBSD/OpenBSDHonorable descendants of BSD

DarwinOpen source OS kernel as a basis of Mac OS X

Other free/open-source OSOpenBeOS (Haiku), Plan 9, GNU/Hurd, etc.

Other OS TrendsMicrosoft's shared source and activities of Sun Microsystems regarding its product Solaris



GNU/Linux

Unix for PC, brainchild of Linus TorvaldsCreated in 1991Bazaar-style development; ported to various platforms ranging from embedded to mainframe

Architectures supported:x86, PPC, Alpha, MIPS, SPARC, S/390, etc.

Strict definition and wider definitionStrict definition: Linux kernelWider definition: Linux package (distribution) with applications

cf. “About Distributions”

Diverse range of GNU/Linux systems popular worldwide



FreeBSD, NetBSD, OpenBSD

BSD : Berkeley Software DistributionUnix-compatible OS created by William Joy(Bill Joy) and Chuck HaleyBased on Unix V6 with networking enhanced

Highly influential on development of today’s Internet-related technologies

FreeBSD/NetBSD/OpenBSDFree/Open-Source OS’es derived from BSD

FeaturesFreeBSD: Emphasis on stability; frequently used for servers, etc.

FreeBSD derivatives: DragonFly BSD, Firefly BSD, etc.

NetBSD: Runs on diverse platformsOpenBSD: Emphasis on security



Darwin

Kernel of Mac OS XReleased by Apple as FOSSMac OS X includes Darwin kernel and GUI (Quartz), APIs (Cocoa, Carbon), etc.

Derived from 4.4BSDMicrokernel: Mach 3.0Released under Apple Public Source License (APSL) version 2.0Supports PowerPC and x86 architectures

OpenDarwin ProjectFounded in April 2002 by Internet Systems Consortium and Apple Computer

http://opendarwin.org/



Other Free/Open-Source Operating Systems

Haiku OS (aka OpenBeOS)Development of BeOS stopped when Be, Inc. was purchased in 2001FOSS version of BeOS; development restarted using released source code

Plan9Next-generation OS developed by Bell Labs of AT&T (now of Lucent Technologies), original developers of UnixBasic design: All resources including CPU are distributed across network

GNU/HurdKernel of operating system based entirely on Free Software; developed by GNU

Aimed at replacing current kernel of GNU/Linux



Other OS Trends

Shared Source InitiativeMicrosoft’s strategic response to FOSSSource code for Microsoft products released through individual agreements with governments, universities and enterprises

Prohibits release of modified source codeNot free to redistribute

Completely removed from FOSS; does not fit definition of FOSS

Developments in Solaris OSSource code released as OpenSolaris in June 2005Some codes only available in binary files; not fully FOSS



Middleware

Web application frameworksMiddleware as a framework for constructing Web applications

JBOSS, Tomcat, etc...

librariesSoftware libraries are enormous assetsExample of libraries

Widget set, graphics library, etc.



Web Application Frameworks

TomcatPart of Apache Jakarta ProjectType of servlet container

Servlet: Mechanism for running Java programs on Web servers

JBOSSImplements Java for J2EE

J2EE: Platform for enterprise Java deploymentSimplifies deployment of Enterprise JavaBeans (EJB)

EJB: Java software component implemented on server side

Developed by JBOSS Inc.



Libraries

glibc (GNU C Library)Collection of most basic and general-purpose components (printf, etc.)Called “C Library” because development on Unix focused on C language

lib*.soShared library

Used by multiple programs

Vast array of libxxx.solibglib, libstdc++, libgtk, etc.

Ex. GNU ReadlineLibrary for editing command lines

Features: History, complementation, etc.

Also provided as libxxx.so: Widget sets, graphics libraries, etc.



Examples of Libraries

Widget setsGTK+(Gimp Tool Kit)

Developed for GIMP graphics editorUsed by GNOME

QtGUI toolkit developed by TrolltechUsed by KDE

OpenMotifGUI toolkit for X Window System; FOSS version of Motif

Mesa (OpenGL)Open Source implementation of OpenGLOpenGL: 3D computer graphics interface developed by SGI

Mono (.NET)Open Source implementation of .NET.NET framework advocated by Microsoft; competes with Java



Servers

Web serverApache

Mail serversMTAML serverPOP3/IMAP

DB serverPostgreSQL/MySQL/Firebird

DNSBIND

File sharingSamba/WebDAV

LDAPOpenLDAP

Mining servernamazu

CMSXOOPS/Zope

Business applicationsE-Learning, e-commerce and many others



Web (HTTP) Servers

ApacheDe facto standard for HTTP servers

72% market share (according to E-soft survey of May 2005)

HistoryDeveloped in 1995 by Rob McCool at NCSADevelopment stalled when McCool later left NCSADevelopers in various places began modifying ApacheThese developers got in touch and started up Apache ProjectName originates from “a patchy” server and the Apache Native Indian tribe

FeaturesLight, fast and reliable (uses modules)Runs on many platforms (Unix, Mac, Windows)

http://www.securityspace.com/s_survey/data/200505/index.html



Mail Transfer Agents (MTA)

SendmailLong used on Unix (developed in 1982)Supports various protocolsMany security holesCommercial version sold by Sendmail, Inc.

PostfixInter-operable with SendmailSimple to configure

qmailFast, robust

No security holes discovered

Simple to configureNot FOSS, strictly speaking (distribution of modified versions is restricted)



Mailing List Server

MailmanWeb-based list administrationImplemented in PythonE-mail archivingBuilt-in attachment file and spam filteringExtensive internationalization support

QuickMLA list created just by sending an e-mailImplemented in RubyAccepts any name for address



POP3/IMAP Servers

qpopperQualcomm’s extension of Berkeley popperAlso supports APOP

UW-IMAPDeveloped by authors of RFC about IMAP

reference implementation



Database Management Systems (DBMS)

PostgreSQLBased on POSTGRES (previously Ingres) developed at UC BerkeleyPioneering object-relational database

MySQLDeveloped by Swedish company MySQL AB

Dual license (GPL and commercial license)Streamlined features and fast performance

FirebirdFree/Open-Source version of InterBase from Borland; released in 2000Name clashed with Mozilla Firebird, prompting Mozilla to rename it Firefox

PostgreSQL top in Japan; MySQL top worldwide



Domain Name System (DNS)

BIND (Berkeley Internet Name Domain)DNS server

System for linking domain names to IP addresses

Developed at UC BerkeleyCurrent maintenance work by Internet SystemsConsortium (ISC)

De facto global standard95% share (based on 2000 survey)

http://www.isi.edu/~bmanning/in-addr-versions.html



Sharing of files

sambaSMB (Server Message Block) protocol

Windows networking uses SMB to implements file and printer sharing

Samba implements SMB services on Unix

WebDAVFile sharing and version management specificationExtends HTTP protocol

Web browser interfaceSupports any OS on client machinesOnly port 80 openedBe secure using SSL and other security features

Supports major Web servers and browsers including Apache, etc



Lightweight Directory Access Protocol (LDAP)

LDAPLightweight Directory Access ProtocolProtocol for accessing directory servicesCentral management of user data, etc.LDAP-compatible software: MS Active Directory, etc.

OpenLDAPOpen Source version of LDAP

Based on SLAPD developed at University of Michigan



Information Retrieval

namazuFull-text search system in JapaneseFeatures

Builds index in advance for fast searchingFilters can be used to search other files in addition to text filesFunctions as a WWW full-text search system when used as CGI

Recent search systemsEstraier: a personal full-text search system

http://estraier.sourceforge.net/

Rast: A full-text search systemhttp://www.netlab.jp/rast/

GNU mifluz (Senga information retrieval software)http://www.gnu.org/software/mifluz/



Content Management Systems (CMS)

XOOPSFeatures

Built using PHP and MySQLSimple to install and build sitesTop FOSS CMS in Japan

Core developers are Japanese (from beginning)

Fork version decided in May 2005, due to dissatisfaction with development structureShift to independent development structure in Japan

ZopeApplication server written in Python

PloneCMS that runs on Zope



Business Applications

Diverse range of FOSS business applicationse-Learning

Moodle, Atutor, FOSS LMS, CFIVE, ...

e-CommerceOsCommerce

Business Server combinesMail-, Groupware-, Web-, Database-Server, Document Management, Anti-Virus, Anti-Spam

ERP (Enterprise Resource Management) and CRM (Customer Relationship Management)

OpenERP, SugerCRM, OSSuite ERP, Compiere, ERP5

Business specific applicationsCerveza restaurant supply procurement system Garagardoa reservation management system



Desktop Environment

GUI and integrated desktop environment

What is a desktop environment?

Mail and web browserMUA (Mail User Agent)Web browser

Document processorEditorOffice suitesType settingPrinting

MultimediaImage processingCG (Computer Graphics)VideoAudio, music

OthersComputation, science and technologyInput of non-latin scripts



GUI EnvironmentX Window System

De facto standard graphics environment for UnixOriginally developed at MIT; widely used today

Core development shifted to XFree86 Project, then to X.Org Foundation

Designed for network transparencyNo distinction between local and remote computing resources

Window ManagerSoftware to control window size, positioning, overlap, etc.Separate from X Windows System itself; installed as standalone applicationInstallation of various window managers

twm, tvtwm, Fvwm, WindowMaker, Enligntenment, Kwin, Metacity, Sawfish, Xfce, etc...



Integrated Desktop Environment

What is an integrated desktop environment?

Provides a common GUI environmentEnables operations involving coordination between applications

Copy & pasteDrag & drop, etc.

GNOMEIntegrated desktop environment based on GTK+

KDEIntegrated desktop environment based on Qt



MUA (Mail User Agent)MUAs using typical three-pane configuration

(Three-pane configuration: Folder tree, title pane and message pane)EvolutionSylpheedThunderbird

MUA derived from Mozilla

MUAs for running within EmacsMewWanderlust

Text-based MUAsMutt



Web Browsers

KonquerorBrowser in KDEIntegrates different media

FirefoxWeb browser from Mozilla project; designed to be light and fast

Other browsersText-based browsers

w3mlynx



Editors

GNU EmacsDeveloped by Richard StallmanExtensible using Emacs LispNot just an editor: platform for text-oriented applications

Vi clonesBased on vi created by Bill JoyVarious vi-compatible installations exist today

vim (vi improved)nvielvis, etc.



Office Suites

OpenOffice.org (OOo)Derived from StarOffice, product of German company StarDivision

Acquired and now maintained by Sun Microsystems

Sun’s strategyPortions of StarOffice not restricted under license are published as FOSS

Comparison with commercial Office software

Comparable function-wise and operation-wise

Full suite of office softwareOOo Writer word processorOOo Calc spreadsheet programOOo Impress presentation toolOOo Draw draw/paint toolOOo Base database program



Typesetting

What is typesetting software?Uses commands to apply a style to a structured document; used for publishingNot WYSIWYG (What You See Is What You Get), but produces high quality output

TeX / LaTeXTeX : Typesetting software created by Donald KnuthLaTeX : Extension of TeX created by Leslie LamportFeatures

Strict concept of style, produces high quality outputSimplifies typesetting of formulasHighly extensible (various extensions are available)



Typesetting (Cont’d)

Typesetting stepsEdit source in TeX (LaTeX)Typeset in TeX (LaTeX) and generate DVI fileCheck typeset results using xdviConvert into PS or PDF filePrint document

GhostScriptRenders PostScript (PS) filesUsed for verification on screen, or used on request by printer driver

xpdfPDF file viewer program



Printing

LPRngng stands for “new generation”

Designed to replace common Berkeley LPR for Unix

Provides security and incorporates modern features, but retains customary interface

CUPS (Common Unix Printing System)Standard print spooler

Issues with UNIX printing systemsLack of common GUI

Ex. Printing dialogs, printer status, etc.

OpenPrinting project under development by Free Standards Group



Image Processing

GIMPPhoto retouching softwareFeatures and ease of use comparable to commercial applications

ImageMagickSet of command-line tools for image processing

Convert image data formatChange size and color gradationVarious special effects

Many other image viewersgqview, eog, ee, gthumb...

Photo retouching in GIMP

convert, identify, composite, montage, compare, display, animate, import, conjure

Command set in ImageMagick



Computer Graphics (CG)

POV-RayRay tracing program for CG creation

blenderProgram for 3D CG creationReleased as FOSS after development company went bankrupt

Can be used with YafRay (Yet Another Free Raytracer)

Open InventorVRML rendering librarySGI software released as FOSS

CG rendering using POV-Ray

Sample program in Open Inventor



Video

xanime, mtv, plaympegFirst wave of video players

MPlayer, XineSupport many video formats

MPEG, AVI, ASF, WMA, QT, MOV, etc.

Continued development in danger due to software patent issue

XawTV, tvtimeTV viewing software relying on TV capture device

Kino, CorianderSaves video from FireWire (IEEE 1394) cameras

Other tools: FFmpeg (video format converter), Ogle (DVD player), etc.



Audio, Music

XMMS (X Multimedia System)Audio file and CD player

LAME (Lame Ain't an MP3 Encoder)MP3 encoder

Ogg VorbisFree music compression formatDesigned to replace other compression formats having many rights issues

RoseGardenDesktop music creation tool

Timidityoftware MIDI synthesizer

XMMS



Computation, Science and Technology

RStatistical computing packageCompatible with S language

SciLab, OctaveScience and technology computing softwareCompatible with MATLAB

MaximaFormula manipulation softwareComparable to Mathematica

Capable of high quality formula display when used with TeXMacs

Graph drawn in gnuplot

gnuplotPlotting software



Kana-Kanji Conversion

FreeWnnFOSS version of Wnn from Omron Software

CannaKana-kanji conversion software developed by NECRestarted in 2002 through volunteer-driven development

AnthyNew entry, developed since 2000; started in response to the dismal state of FOSS-based kana-kanji conversion software

Issues with kana-kanji conversion wereNo standard common framework

Candidates: XIM, IIMF and UIM protocols

Poor performance due to patents covering conversion methods



Input of Non-Latin Scripts

Increasingly users require non-latin scriptsGoal: provide common frameworkExamples: Kanji, Chinese, Hangul, Cyrillic

but also for German, French, Spanish, ...

Smart Common Input Method (SCIM)full featured input method user interfacefor POSIX-style operating systems

Linux, FreeBSD and other Unix

development platform for input methodscurrently supports more than 30 languages



Development Environments

LanguagesVarious programming languages are provided as FOSS

Integrated development environmentsEclipse: the most famous FOSS IDE (Integrated Development Environment)Other IDEs: KDevelop, Anjuta, WideStudio, etc.

Development frameworksStruts is quoted as a typical example of a framework which can improve development efficiency and average quality of products



Developing Languages

gcc (GNU Compiler Collection)

Collection of compilers for C, C++, Fortran, Java, etc.Standard compiler for development on Unix

PerlStrong text processingFlexibility to use various syntax for same process

TMTOWTDI: There's More Than One Way To Do It.

Frequently used for system management and CGIs

PHPHTML-embedded, server-side scripting languageMain language for Java and Web system development

LAMP/LAPP

PythonFeatures block designation using indentation

RubyDeveloped by Yukihiro Matsumoto

Python and Ruby are rival languages

Python is an object-oriented version of Perl



Integrated Development Environments (IDE)

EclipseDevelopment environment implemented in JavaSupports languages other than Java

Plug-ins for C/C++ developmentC/C++ Development Toolkit (CDT)

IDE for various desktop environments

Kdevelop for Qt/KDEAnjuta for GTK+/GNOME

Other IDEWideStudio

For creating GUI applications using C/C++



Development Frameworks

FrameworkImplements basic framework for data I/O, error handling, screen transitions, etc.

Improves development efficiencyStandardization of quality: Same quality regardless of who writes it

Difficult to implement features outside of framework

StrutsJava-based Web application framework

Uses Java servlet and JSP technologyUses MVC architectureSimple and powerful frameworkRuns on servlet container such as Tomcat

Developed by Jakarta Project



Major GNU/Linux Distributions

About distributionsCollection of software

RedHat / FedoraTop share for enterprise Full of leading-edge functions

Turbo-LinuxFocus: desktop

UbuntuBased on DebianFocus: desktop

SUSE LINUXDistribution originally from GermanyThe second largest share of GNU/Linux market

Vine LinuxWell-tailored Japanese environment

Debian GNU/LinuxMany developers in the world

KNOPPIXBootable from CD



About Distributions

GNU/Linux distributionsStandard applications do not run on Linux kernel aloneRequires a shell, libraries, tools, etc.

Also requires application software

Distributions bring together necessary software to simplify installation

Can be installed by novices

What sets a distribution apart?Different applications or configurations for server/desktop useInclusion of commercial software

Commercial-free versions often available for free download

Package management systemsrpm, deb, Portage

Boot method (hard drive, CD or floppy)



Red Hat Linux, Fedora Core, CentOS, etc.

Red Hat LinuxFrom Red Hat, the largest GNU/Linux distributor

Red Hat focus is on enterprise applications

Development of free Red Hat Linux stopped in 2003FOSS development shifted to Fedora Project

Clone distributions excluding commercial softwareWhite Box Enterprise Linux, CentOS

Fedora CoreCommunity-based development under Fedora Project

Supported by Red Hat

Aggressively adopts advanced featuresAlso serves as Red Hat’s test versionUses UTF as default character code

Some conflicts with existing software



Turbolinux

*1 IDC, “China Linux 2004—2008 Forecast and Analysis,” July. 2004*2 IDC Japan, “Operating System Market in Japan: Linux Market Analysis and Forecast 2003—2007” (In Japanese)

Developed and marketed by Japanese company Turbolinux, Inc.

Japan: No. 2 in GNU/Linux server OS market share (2003 survey) *1China: No. 1 in GNU/Linux server OS market share (2003 survey) *2

Released Chinese version of GNU/Linux

Joined UnitedLinux industry consortium in 2002Purpose: Establish standard distribution for GNU/Linux

Focus on desktop GNU/Linux OSReleases: Turbolinux 10 Desktop, etc.

Enhanced with Windows-like GUIComes with player supporting Windows Media video files



SUSE Linux

Originally developed by German company SUSE Linux

No. 2 in global market shareNo. 1 in Europe

Acquired by Novell in 2004OpenSuse: contains only FOSS

FeaturesAdopted KDE as standard desktop environmentFirst enterprise GNU/Linux distribution to use Linux kernel 2.6Includes YaST integrated management tool



Vine Linux

Community-based development by Project Vine in Japan

Commercial version available from Vine Caves, Ltd. with additional fonts and kana-kanji conversion

FeaturesJapanese language environment is easy to useEmphasis on stability

Slow to adopt new features

Employs apt package manager (compatible with RPM)



Debian GNU/Linux

Developed by Debian ProjectDebian Project also develops other free operating systems outside of GNU/LinuxEncompasses nearly 1,000 developers

FeaturesUses apt and deb package managersVast number of packages (over 15,000)Entirely made up of FOSS

Various distributions derived from Debian GNU/Linux

KNOPPIXLinspire: Windows-like commercial OSSkolelinux: GNU/Linux distribution for schools



KNOPPIX

Developed by Klaus Knopper derived from Debian

FeaturesBoots from CD-ROM

Easy to try out; does not affect existing environmentCan be installed to hard drive

Excellent automatic device detection



Ubuntu

Based on DebianStarted by Mark ShuttleworthCommercial support and development by Canonical



(4) Philosophy and History of Free/Open-Source Software

History of FOSSLooking round the evolutionary process of FOSS from “Birth of Hackers” to current situations where FOSS is widely spread in enterprise market

Definition of FOSSUse the term FOSS correctly by knowing strict definition of “Open Source Software”

FOSS LicensesTo understand meanings of software licenses and categories of licenses



History of FOSS

Birth of hackersBasis of emerging free software

Free software movement and FSF

Free software as freedom of software

CopyleftFundamental structure to keep freedom of software

Spread of the InternetFree software played an important role

Birth of LinuxA pioneer of “Bazaar model development”

Growth of FOSS, and Launch of OSI

Market share of FOSS has enhanced for business use

Penetrating the enterprise market

Many major computer vendors join the FOSS enterprise market



Birth of Hackers (1950s to Present)

“Hacker” used at MIT to describe a person who writes good programs or new algorithms

Individuals who attack systems are “crackers”

Hacker culture is a type of gifting culture

“Hacker” title bestowed by others

Unix conceived in 1971

“Hack” (definition from thefreedictionary.com)

Originally, a quick job that produces what is needed, but not well.

An incredibly good, and perhaps very time-consuming, piece of work that produces exactly what is needed.

To bear emotionally or physically.

To work something (typically a program).

To pull a prank on.

To interact with a computer in a playful and exploratory rather than goal-directed way.



Free Software Movement and FSF (1980s)

Aim: World where citizens can live with Free Software aloneGNU Manifesto (1983)

Author: Richard Stallman

Launch of GNU Project (1984)Develop Unix-compatible OS from scratchDeveloped main Unix functions through 1990

Free Software Foundation (FSF) (1984)Founder: Richard StallmanOrganization for managing copyleft softwareGNU General Public License (GPL) V. 1 released (1989)GPL Version 2 released (1991)GPL Version 3 released (2007)



Copyleft

CopyleftStipulates freedom of program and freedom of modified or extended versions

Protects freedom of software from copyrightDoes not abandon copyrightManages copyright and ensures freedom of license

Right of copyright holder cannot be abandoned under e.g. Japanese or German law, preventing notion of public domain software



Spread of the Internet

ARPANET launched by US Department of Defense (1969)

NSFNET branched off from ARPANET (1986)Network for research communityLinked computers at research institutes worldwide

Free Software drives growth of InternetE-mail (Sendmail), DNS (Bind), newsgroups (INN), etc.

Internet opened to commercial interests (1991)

NCSA Mosaic Web browser conceived (1993)Impetus for Internet boomGrowth of online e-commerce (since late 1990s)



Birth of Linux (Early 1990s)

Linux released by Linus Torvalds in October 1991

Student at Helsinki University in Finland

Licensed under GPLFree Software success story

Pioneered bazaar-style development modelFrequent beta releases in mid-developmentMany developers send bug reports and patches

Driving factor in emergence of Free/Open-SourceBusiness success of Linux drew attention to effectiveness of bazaar-style development and its software, but Free Software movement was too radical for companies



Growth of Free/Open-Source; Launch of Open Source Initiative (Late 1990s)

OSI (Open Source Initiative)Founded by Eric Raymond and Bruce PerensStrategic initiative to promote adoption by businessRelease of Netscape source code impetus for founding OSI

“The Open Source”Proper noun focusing on software development modelPrescribed under the Open Source DefinitionIncreased recognition has led to greater misuse and misunderstandingOther terms being considered

OSS : Open-Source SoftwareFLOSS : Free/Libre/Open-Source Software



Penetration of Enterprise Market (2000s)

IBM begins offering GNU/Linux support (1999)

Open Source Development Labs (OSDL) founded (2000)

GNU/Linux development for telecommunications, data center and enterprise desktop applicationsLinus Torvalds joins OSDL (2003)

Consortium formed by IBM Japan, Hitachi, Fujitsu and NEC (2001)

Development to extend functionality of enterprise GNU/Linux

GNU/Linux deployment spreads to enterprise backbone systems and financial institution systems



Definition of FOSS

The software whose source code is published, does NOT equal to FOSS

Insufficient, if ONLY publishing its source code

GNU's definition“Free Software”

OSI's definition“Open Source Software”



GNU Definition

“Free Software” is:Software with the freedom to run, copy, distribute, study, change and improve the software

The freedom to run the program, for any purpose.The freedom to study how the program works, and adapt it to your needs.The freedom to redistribute copies so you can help your neighbor.The freedom to improve the program, and release your improvements to the public, so that the whole community benefits.

Abstract shape: CopyleftConcrete shape: GNU General Public License (GPL)

OK to charge for distributionNot all Free Software asserts copyleft



Definition by Open Source Initiative

Prescribed by the Open Source Definition (OSD)

Wider recognition leads to greater misuse of termCriteria for FOSS licensesOver 50 licenses approved

Based on Debian Free Software Guidelines (DFSG)

OSD places greater emphasis on distribution criteria

Free Software complies with Open Source definition

Free RedistributionSource CodeDerived WorksIntegrity of The Author's Source CodeNo Discrimination Against Persons or GroupsNo Discrimination Against Fields of EndeavorDistribution of LicenseLicense Must Not Be Specific to a ProductLicense Must Not Restrict Other SoftwareLicense Must Be Technology-Neutral



FOSS Licenses

Software licensesTo begin with, what is a software license?Click-on contract is frequently used

Typical FOSS licenses and their characteristicGPL/LGPLBSDOther licenses, dual licensing

OSI-approved licenses



Software Licenses

Sales format for software actually for:Media costs for distribution?Commensurate cost of contract development?

Right to use software is generally purchasedLicenses

Types: Site license, volume account license, etc.

Licensing is an act of contractFrequently takes shape as contract of adhesion

Shrinkwrap contractUser enters into agreement when shrinkwrap on package is broken

Click-on and click-wrap contractsUser enters into agreement when button is clicked in contract dialog box



Example of Click-On Contract

EULA (End User License Agreement)Appears first time application is startedUser enters into contract when Accept button is clicked



GPL

GPL expresses copyleft as concrete license

GPL (GNU General Public License) GNU Public License (GPL)If derivatives of GPL’ed software are distributed, it must be accompanied by source codeProtects freedom of software

DerivativesModified source codeUses GPL’ed libraries

GPL v3 protects software freedom from PatentsMechanisms to prevent modification



GPL and LGPL

LGPL (GNU Lesser General Public License)GNU Lesser General Public License (LPGL)

Formerly “The GNU Library General Public License”

Features of LGPL (derivative licenses)Derivatives based on modified source code must also conform to LGPL

No license restriction for software using LGPL’ed libraries through dynamic linking

No clear line between derivative and non-derivative work, depending on extent of linkingGNU’s position: Work is not a derivative if interface is clearly defined and software module split off

LGPL a product of compromise?LGPL a departure from GNU idealsSoftware that links to GPL’ed libraries must also conform to GPL, which could discourage useMany libraries use LGPL



BSD

Berkeley Software Distribution (BSD) LicenseBSD ... Berkeley Software DistributionMuch less restrictive

No protection of software freedom

Derivative work can be turned into proprietary softwareAdvertising clause removed in June 1999

Modified BSD LicenseOmits advertising clause:

“All advertising materials mentioning features or use of this software must display the following acknowledgment. This product includes software developed by the University of Carifornia, Berkeley and its contributions.”



MPL, Apache License, Dual Licensing

MPL (Mozilla Public License)Mozilla: Open-Source Web browserPermits closed license for standalone software (applications, plug-ins)Contains references to legal jurisdiction and patent issuesIncludes escape clauses for patent infringement

Apache License from Apache Software FoundationLicense used for Apache Project (Web server project)Includes advertising clauseNot compatible with GPL

Multiple license formatsDual licensing, triple licensingEx. Qt (QPL or GPL), MySQL (GPL or commercial license), Mozilla (MPL, GPL or LGPL)



OSI-Approved Licenses

OSI certification markOnly to be displayed for software suitably distributed according to OSI-approved license

OSI-approved licensesDenotes conformance with OSD; requires application to OSI for license approval58 licenses (as of June 30, 2005)

http://www.opensource.org/licenses/index.html

GPL-like licensesGPL

MPL-like licensesMPL, LGPL, CPL, Artistic License

BSD-like licensesBSD, MIT, ASF



(5) FOSS Development and the FOSS Community

FOSS and the FOSS community

FOSS is closely-linked to activities in the FOSS community

Development structureHow FOSS project works?How to participate in FOSS projects?

Topics related to FOSS development

CommunicationFOSS projects are supported by communication between development participants

FOSS developersFLOSS survey: FOSS developers on-line survey, conducted from 2002 to 2004



FOSS and the FOSS community

FOSS Distribution: Community to UsersJapan FOSS promotion forum's working group made a detailed comment on FOSS distribution from communities to users

Types of FOSS communitiesHow developers and users gather to form their community?

User communitiesGrowth of mutual support culture due to a lack of vendor supportThe next issue is to make a good relationship between user community and developer community



FOSS Distribution: Development Community to Users

Report prepared by WG of Japan FOSS ForumFindings of Support Infrastructure Working Group

Describes mechanism for FOSS distribution from development community to users

Published: February 10, 2005

Contents of reportGNU/Linux distributions: Main FOSS distribution route to usersOrganizations central to GNU/Linux distributions

Development community; user support options and range of support

Summary and major points to consider for FOSS deployment

Member corporations of Support Infrastructure WG:NEC, NTT Comware, NTT Data, OSDL, NS Solutions, Turbolinux, Nihon Unisys, Novell, Nomura Research Institute, Hitachi, Fujitsu, Miracle Linux, Red Hat



Types of Communities

Developer communitiesCommunities formed around developer groupsMajor FOSS projects form large-scale communitiesMembers take on various roles

User communitiesUser communities formed for each applicationCooperative role to address lack of information

Use mailing lists and message boards

Local Linux User Groups (LUGs)Scores of LUGs across Japan



User Communities

Prolific number of user groups, preceded by name of FOSS

e. g. Top 10 results of Google search using Japanese keyword User-kai (June 2005):

Japan PHP User GroupJapan MySQL Users GroupSamba Users Group JapanJapan PostgreSQL Users Groupja: Japan: OpenOffice.org Japan Users GroupJapan Apache Users GroupJapan Zope User GroupJapan GNOME Users GroupJapan UNIX SocietyPython Japan User's Group

Following results: User groups for KDE, Mathematica, Analog, Snort, SELinux, Firebird, etc.



Development Structure

Cathedral and Bazaar-style developmentComparison between two types of development methodBazaar-style development was spotlighted starting with great success of Linux

Roles of project membersUnderstanding the roles of various project members

Development resourcesServices and businesses that support FOSS developmentExamples of the service

Project repositories

Getting involved in projectsManagement of FOSS projects



Cathedral and Bazaar-Style Development

Eric Raymond’s 1997 essay The Cathedral and the Bazaarhttp://www.catb.org/~esr/writings/cathedral-bazaar/

Cathedral modelGNU described as example of cathedral development, predating Linux

Note: Cathedral model is not synonymous with development style of proprietary software

Only specific participants involved in developmentNo releases until specific results are achieved

Bazaar modelLinux development used as example of bazaar-style development

Open group of participants; respect for individuality; talented coordinator manages flow of projectReleases any time in mid-development; basic notion of “fast and frequent releases”

Bazaar-style development since used extensively for FOSS development projects



Roles of Project Members

Project leader: Authority to direct projectCore members: Authority to change source code

Project participantsRegular developers: Provide feedback about new features, enhancements and bug fixesTesters (advanced users): Test software and report bugs for each releaseDocument authors: Create documents for general users and developers

Other roles:Resource development (non-programming)Evangelism, PR (mailing list and/or Web site management)Financial supporters and providers of development resourcesSupporters for offline activities



Development Resources

Project hosting servicesFree/Open-source project repositories

SourceForge.netBerliOSFSF/UNESCO Free Software Directorysavannah.gnu.org

Software directory sites, software search enginesFreshMeat.netRpmfind.net, others

Other development resourceswww.koders.com



SourceForge.net

Operated by OSTG, Inc. (Open Source Technology Group)

http://sourceforge.net

ServicesProject Web site hostingCVS repositoryBug tracking systemCommunication toolsOthers services

Japanese versionSourceForge.jp



BerliOS, Free Software Directory

http://www.berlios.de/ http://directory.fsf.org/



FreshMeat.net, Rpmfind.net

http://freshmeat.net/ http://rpmfind.net/



Getting Involved in Projects

Find software that implements functions you wantNumber of projects available online is nearly unlimited

Debug or extend functionality of softwareIf you find a function that doesn’t work or is missing

Take full advantage of released source code

From feeding back your results to joining a development team

Get involved to advance software science and the FOSS community

Find existing projects through a project repositoryMake active use of development resources



Tips for Getting Involved

Honest and factual reportingImportant to provide objective data when reporting bugs

ReproducibilityConfiguration, what you were doing, and what happenedInclude error messages, logs

Code you contribute is not just for youObserve certain courtesies

Adhere to coding rulesInclude comments so others can read and understand your code

Include a descriptionNever send just your code (worst thing you can do)Use simple and concise sentences to describe your code



Project Management

Personal relationships are criticalWorldwide development

Enables round-the-clock development: requires use of various communication tools

Progress of project accelerated by contributions from othersRequires communication skillsRequires proper assignment of roles based on contributions to development

Share source code through the InternetSourceForge.net lists 1,000,000 developers and 100,000 projectsAccelerate development through frequent releases and quick revisions

Does not ensure quick completionEnsure high quality and strong security through “eyes of many”



Topics Related to FOSS Development

Agile development and similar methodologiesSome similarities are there between agile development and FOSS development

Stable versions and development versionsTwo versions are simultaneously released: stable version for average users and development version for developer and progressive users

Project forksThere might be cases that project is divided by two opposing points of view about its directionSource code tree is branched at some point

Advantages and Downsides of FOSS

development



Agile Development and Similar Methodologies

Twelve core practices of Extreme Programming (XP):1. Planning Game Continually plan in small units

2. Small Releases Short release cycles

3. Metaphor Use metaphors for ease of understanding

4. Simple Design Keep the design simple

5. Testing Prioritize testing in program development

6. Design Improvement (originally: Refactoring) Actively review code

7. Pair Programming Programmers write code in pairs

8. Collective Code Ownership Make all codes accessible to every programmer

9. Continuous Integration Continually test and keep the software running

10.Sustainable Pace (originally: 40-hour Week) Overwork leads to lower productivity

11.Whole Customer (originally: On-site Customer) Incorporate users into the team

12.Coding Standards Program out of respect for coding conventions

2, 8, 9, 12 are also important in FOSS developmentHowever, No. 1, 3, 7 and 10 are difficult to implement under FOSS development



Stable Versions and Development Versions

Bazaar modelPolicy of fast and frequent releases

Conservative users inconvenienced by trial-and-error approach to new features

Releases separated into stable versions and development versions

Development versions: Source code heavily modifiedFor developers and cutting-edge users

Addition of new features

Odd version numbers: 1.1->1.3->1.5…

Stable versions: Changes kept to minimumFor general users

Incorporates bug fixes, security patches, etc.Well-received features sometimes back-ported from development versions without waiting for major update

Even version numbers: 1.0->1.2->1.4…



Project Forks

On rare occasions, projects can branch offExamples of project forks

XFree86 -> X.OrgEmacs -> XEmacs

Sometimes projects fork and merge againEmacs -> Mule -> EmacsGCC -> EGCS (Experimental/Enhanced GNU Compiler System) -> GCC

FOSS helps to maintain sustainability of development but also allows project forks to occur

x-1.0 x-1.1 x-1.2 x-1.3 x-2.0 x-2.1

y-1.0 y-1.1



Advantages of Free/Open-Source Development

Fundamental advantages of program development on FOSS platforms

Low cost (although Free Software does not mean free of charge)Comes with source codeHigh scalability (everything from embedded to mainframe)

Use existing Unix technologyAbility to use Unix technology and existing software assets on low cost hardware (PC-compatible machines)

Increasing availability of development tools for various applications

Mission critical fieldsImage processing, multimedia, science and technology computing, etc.Embedded applications



Risks of Free/Open-Source Development

Shortage of FOSS engineersDemand outstripping supply (both for user enterprises and system integrators)Urgent need for human resources development programs, training environments, training materials and university education

Retraining of Unix engineers offers shortcut

Lack of guarantees and supportShift needed from volunteer development to enterprise supportSupport issue highly dependent on technical capabilities of user company

Lack of experienceNewness of FOSS development modelKnowledge of how to collaborate with development community still to comeChallenge of finding balance for securing intellectual property rights



Communication

Who develops software?Communication is critically important

Communication toolsOn-line communication utilizing various toolsWhat kind of tools are used?

Pros and cons of on-line communication toolsEvaluating the pros-and-cons of each tools

Off-line communicationImportance of off-line communication

Participating in conferencesCommunicate actively via off-line meeting



Communication Tools

Methods of online communicationE-mail exchange between developersMailing lists administered by project

Developer mailing list: [email protected] mailing list: [email protected]

Message board on project Web siteIRC (Internet Relay Chat) and other chat systemsWiki, developer blogs

Conventional tools to supplement online communication

Telephone, fax, etc.

English is the universal language of communicationOverall communication skills more important than English ability



Online Communication Methods: Pros and Cons

Pros Cons

E-mailFamiliar user interface

Mailing list Familiar user interface

Wiki

Blog

Not suited to discussion between multiple persons

Redundancy from copies of all messages

Message board

Accessible, shows flow of discussion

Discussion tends to become derailed

IRC and other chat systems

Allows for instantaneous discussion

Not suited to in-depth discussions

Emphasis on documentation

Comparatively static medium

Blogger can assert own opinions

Not designed for discussion



Offline Communication

Power of offline communicationSignificance of face-to-face communication

Non-verbal communicationBody and hand languageExpression, tone of voice, general mood, etc

Sometimes accelerates development through kinship and understanding personalities of others

Limits of online communication felt during:Eruption of flame warsObservance or non-observance of netiquetteDifference between written and spoken words

Difficult to convey subtle nuances through writing

Could be that social gatherings are most important



Participating in Conferences

FOSS-related global conferencesLibre Software Meeting/ Rencontres Mondiales du Logiciel Libre (France) LinuxTag (Germany)Ottawa Linux Symposium (Canada)Many others such as DebConf, BSDCon, etc.

Actively interact with othersAdvantages

Get word out about own projectGet advice about software design and implementation

Could lead to cooperation with developer teams

Mutual exchange between developers and usersBuild personal network, exchange information, etc.



FOSS Developers

FLOSS Surveys

Developer background and InvolvementWhen FOSS developers get involved in their development?

Reason for getting/staying involvedWhat is the reason to participate in FOSS activities?

Type of involvementWhat is their involvement in FOSS projects?

Global involvementRealities that global communications on FOSS activities

CompensationDo they receive money in return for their work?



FLOSS Surveys

What is a FLOSS survey?Provides insight into conditions of FOSS developers

Analysis-based online survey

Free/Libre/Open Source Software online surveyFLOSS Survey (2002)

Conducted by Maastricht Economic Research Institute on Innovation and Technology (MERIT)

FLOSS-US Survey (2003)Conducted by Stanford Institute for Economic Policy Research (SIEPR)

FLOSS-JP Survey, FLOSS-ASIA Survey (2003-2004)Conducted by Mitsubishi Research Institute, Inc. (MRI)

Commissioned by Japanese Ministry of Economy, Trade and Industry (METI)



Free/Libre/Open Source Software Japanese Developers Online Survey

FLOSS-JP Survey

Survey aimed squarely at FOSS developersPeriod: September 1, 2003 to November 1, 2003Online survey and questionnaires distributed at conferencesValid responses: 547

Survey contentFOSS/FS experience, level of involvement, personal profileInitial impetus for getting involved in FOSS/FS development, reasons for staying involved, personal viewsIncome from FOSS/Free Software development, sponsorship income, etc.

For more about the survey, visit the URL below:http://FOSS.mri.co.jp/floss-jp/



Developer Background and Involvement

Personal backgroundLate 20s/early 30s, mostly maleMajority possess undergraduate or master’s degreeMany software engineers, programmers, students and university instructors

Position on FOSS or Free SoftwareIdentify with FOSS: 43.7%Identify with Free Software: 26.7%Do not care: 29.6%

Time spent on development (per week)

Two-thirds spend 5 hours or less



Reason for Getting Involved

Initial reason for getting involved in projectReleased program I wrote myself: 36.5%Sent in patch: 16.2%Exchanged e-mail with author: 10.7%Replied to user’s question on mailing list: 10.5%Reported bug: 10.5%



Reasons for Staying Involved

Own motivation for continued development (multiple responses allowed)

Learn new skills: 64.9%Share knowledge and skills: 48.9%Solve problems not possible with proprietary software: 29.2%Improve FOSS/FS from other developers: 24.9%Realize new idea for software: 22.5%

Two-part questionOwn motivationOpinion on motivation of others



Type of Involvement

Main types of involvement (multiple responses allowed)

Develop main functions: 52.1%Write bug fixes: 36.0%Write patches: 33.6%Testing: 20.9%Document translation: 14.3%Document preparation: 14.0%Packaging: 12.5%Localization: 10.8%Project management: 10.2%User support: 9.8%



Global Involvement

37.7% involved in global development community

Mainly active in Japanese community: 62.3%Mainly active in global community: 16.1%Involved in both: 21.6%

English ability68.3% feel own ability is limitedEnglish ability and level of activity

Not necessarily relatedDepends on enthusiasm and patience



Compensation

Only 26.8% of developers receive direct income from FOSS activities

41.2% have profited indirectly from FOSS activitiesHired or found new job due to FOSS development experience: 9.6%

Data indicating social recognition as technical experience

Awareness of involvement in FOSS development

School/employer not aware of own involvement in FOSS: 40.8%Preference: Desire to be involved in FOSS development as part of work



(6) FOSS Business

Reasons for interest in FOSS business

Relation between changes of IT environment and FOSS business

Factors driving FOSS business

How is the market trend?

Types of system development using FOSS

Examining typical FOSS systems

Entry of Major Vendors into FOSS business

Many manufacturers are moving in FOSS business market for a variety of reasons



Reasons for Interest in FOSS Business

Changes in IT environmentEnormous improvement of S/W, H/W and highly sophisticated network infrastructure (wide use of the broadband Internet)

Changes in the software businessSoftware vendors have to change their business model

Shift from package sales to servicesMarket requires IT vendors to change their business to service-oriented business



Changes in IT Environment

Advances in hardware and implementation of software

Faster CPU speedsMoore’s LawCPU clock frequencies nearing their limits

Trend in technology development shifting to parallel and distributed computing

Advances of software technologyP2P technologyMultimedia technologyAdvances of software design methodologies

Spread of Internetbroadband (ADSL) connectivity

Now possible to freely download large-scale software



Changes in the Software Business

Need for transformation of software business, driven by rapidly shifting IT environment

Conventional software businessPackage salesApplication developmentSystem integration

New issues emerged by changes in IT environmentWidespread unauthorized use of illegally copied softwareIncreasing complexity of software

Numerous security incidentsRapid obsolescence

What kind of business model is best?



Shift from Package Sales to Services

Shift to service-oriented businessEmergence of ASP model

Provide services over Internet, using Web applications

Does not concern with its implementationUse FOSS to enable early and low cost services

Other service-based businessesCustomizationSystem integration using FOSSSelective use of FOSS in application development for improved efficiency



Factors Driving FOSS Business

Expanding GNU/Linux related marketGNU/Linux has already been in use in many fields of practical business

Network servers, mission-critical systemsComputational applications, desktopsEmbedded systems

Case study: the situation of Japanese market



Growth of GNU/Linux Market

Internet serversWeb, e-mail, DNS, firewallsAlready established in this field

Work group serversFile sharing, client-server systemsDeployment rapidly increasing as segment enters growth period

Servers for backbone systemsHigh reliability servers for large-scale tasks

Finance, telecommunications, e-government, etc.Studies by OSDL’s Carrier Grade Linux (CGL) working group, etc.Serious deployment began in 2003



Growth of GNU/Linux Market (Cont’d)

Science and technology computingSimulations, analysis, data miningGNU/Linux share rapidly growing; field naturally suited to GNU/Linux

>85% of Top500 run GNU/LinuxTop 5 all run GNU/Linux

Linux Networx (2300CPU, 7.6TFlops)

GNU/Linux used lately for grid systems research

Desktop GNU/LinuxMajor factors: Spread of FOSS office suites, low cost PCs, etc.Many issues to clear before spread of GNU/Linux in Asia

Embedded GNU/LinuxDedicated devices for specialized applications: hard disk video recorders, Sharp Zaurus PDA, OpenMoko phone



GNU/Linux Market in Japan

2003 2004 2005 2006 20070

100

200

300

400

500

48 55 70 90 11055 63

80

100

130

8094

120

160

210

Forecast for Growth of Linux Business in Japan

Linux SISupportLinux PF(HW,SW)

Source: NEC Corporation

BillionJapaneseYen



GNU/Linux Environments (World)

2004 2005 2006 2007 2008 2009 2010 2011 20120

500

1000

1500

2000

2500

3000

3500

4000

4500

712 8631091

1363 1554 1709 1863 2012 21331132

12701281

13001353

14481564

16891816

Forecast for Growth of Linux Environments Worldwide

non-paidpaid

Source: IDC, April 2008

Thousands ofdeployments



Types of Systems Development Using FOSS

Market penetration and reasons for FOSS system adoption

More than half of small-scale servers adopt FOSSMajor reasons are relatively lower cost and stability of FOSS

Typical FOSS system structuresLAMP/LAPP systemsJava-based systemsConventional applications for FOSS servers



Market Penetration and Reasons for FOSS System Adoption

Adoption of FOSS systems on small-scale serversWhere are FOSS systems used?

(Source: Survey by Nikkei Business Publications)Web servers: 60%Work group file servers: 50%

Reasons for adopting FOSS systemsLow cost: 65%Stable operation: 50%Security: 30%Ease of remote operation: 25%



LAMP and LAPP Systems

Web service development accounts for large percentage of Free/Open-Source development

Ranks high in FLOSS-JP Survey

Typical system configurationsLAMP systems development

GNU/Linux OS, Apache Web server, MySQL database, PHP script languageLAMP stands for Linux, Apache, MySQL, PHP

LAPP systems developmentGNU/Linux OS + Apache Web server + PostgreSQL database + PHP script languageLAPP stands for Linux, Apache, PostgreSQL, PHP



Java-Based Systems

Lower development costsUse Free/Open-Source Tomcat in place of commercial servlet enginesSide benefit of using Java: reduced OS dependency

System configurationGNU/Linux + Apache + Tomcat

Comparatively small-scale systemsUsed to implement systems comparable to LAMP systems

Large-scale systemsExtensive use of J2EE or frameworksUse GNU/Linux for running environment

Frequently rely on commercial components to implement complex business logic



Conventional Applications for FOSS Servers

Information sharing services, operation of portal sites

Apache used for Web serverVarious modules can be used to provide diverse services

Small-scale system integrationWork group server: Combination of standalone FOSS servers provides adequate service level

Use of package LAMP systemsCMS, business application packages, etc. Closer inspection frequently reveals LAMP or LAPP system underneath



Major Vendors Moving into FOSS Business

On-demand computing and FOSS

IBM's GNU/Linux branding strategy

Situation for leading Japanese manufacturers



On-Demand Computing and FOSS

Mainframes -> Open systems (client-server) -> Web computing -> ?Vision: networked servers provide on-demand computing power

Computers used like water or electricityMuch like turning a faucet produces water or plugging into an outlet provides electricity

No need to worry where water or electricity comes from

IBM's on-demand computing, Sun's utility computing

FOSS-based platform for on-demand computingCompete through services, not infrastructure or packagesReasons FOSS is suited for common platform: low vendor dependency, modifiable by each company, low cost, scalabilityIBM strategically positioning itself behind FOSS in anticipation of transformation into services company



IBM’s GNU/Linux Branding Strategy

IBM views itself as trend-setter for computer industry

Pride as industry leader for over 30 years

IBM correctly predicted that GNU/Linux would steadily gain popularity

However, GNU/Linux cannot be owned by any companyFear that IBM might lose its advantage

IBM decides to back GNU/LinuxPrevent GNU/Linux from going in its own directionBuild brand image where GNU/Linux is synonymous with IBMCan be interpreted as attempt to take over GNU/Linux brand



Situation for Leading Japanese Manufacturers

Revenue source disappearing for leading Japanese computer makers

Shrinking market for mainframes (except for IBM)Shift from domestic Unix to overseas Unix

Increasingly difficult to maintain proprietary OS1,000-3,000 engineers needed to develop and maintain single OS

With GNU/Linux, core technology can be maintained with just 100 engineers

Japanese computer manufacturers joined OSDL Japan in 2001As FOSS, GNU/Linux offers low risk of direct influence from other companiesWith backs to wall, no choice but to bet on GNU/Linux



(7) Case Studies in FOSS Business

Types of FOSS business

FlagshipSystem integrator

NTT Comware, Nihon Unisys, etc.Case studies in large-scale development using FOSS

DistributorBusiness strategy of distributors

In-house FOSS

FOSS specializationPackage softwareServer salesEmbeddedEmbedded development toolsIn-house business useEducation



Types of FOSS BusinessCategory Description Leading Companies

Flagship

System integrator Contract development using OSS NTT Comware, NEC Soft, etc.

Distributor

In-house OSS Ten Art-ni, NaCl, ForeOneFirst, etc.

OSS specialization

Package software

Server sales Sales of Linux-equipped PCs and servers Dell, Toshiba, Sun Microsystems, etc.

Embedded

Use OSS for own business

Education

Actively provide everything from contract development of systems using own hardware and OSS, to software and hardware maintenance and support

IBM Japan, Fujitsu, NEC, Hitachi, HP Japan, etc.

Packages sales of Linux with various types of OSS, commercial software, installation support, etc.

Red Hat, Turbolinux, Miracle Linux, etc.

Release own software as OSS; provide support services and contract systems developmentSpecialization in specific OSS; provide support services and contract systems development

VA Linux Systems Japan, SRA, Zend Japan, etc.

Sales of Linux-compatible versions of own package software; also provide contract systems development

Oracle, NIWS, Horizon Digital Enterprise, F-Secure, etc.

Development and sales of OSS-embedded information appliances, PDAs, mobile phones

Sony, Matsushita (Panasonic), Toshiba, Sharp, etc.

Embedded development tools

Customization for OSS embedded use; provide development tools and support services for fee

MontaVista Software Japan, Lineo Solutions, AXE, etc.

In-house business use

Amazon, Google, Rakuten, Livedoor, etc.

Provide exams and training courses for OSS qualification and certification

LPI-Japan, Red Hat, Linux Training Institute, etc.



Flagship Business(Major Manufacturers)

Receive comprehensive orders for everything from hardware to softwareBusiness of major hardware vendors

IBM Japan, Fujitsu, NEC, Hitachi, HP Japan, etc.Use FOSS to drive down costs for development of large-scale systemsProvide comprehensive solutionsPossess numerous own development packages

IBM in particular strongly pushing FOSS Provides FOSS resource site and FOSS informationNumerous in-house FOSS projectsSupports GNU/Linux through full range of platforms, from IBM PCs to mainframes and wearable PCsReleased 500 IBM patents to FOSS community

Reason for market participation by major hardware vendorsSystem integration anticipated to generate annual profit of US$1.1 billion worldwide



System Integrators

Extension of existing business, only platform has changedDrive down development costs using PC servers and FOSSAlso reduces vendor dependency

Enables small- and medium-sized vendors to compete as principal contractors, but competition increasingly fierce

Biggest hurdle is securing FOSS engineersFOSS deployment requires advanced technical skills

Requires skills to combine multiple FOSS, modify FOSS, keep up with fast pace of change, etc.

Easier to retrain Unix engineersProblem: Many engineers only know Windows development

Shortage of business applicationsDevelopment above LAMP level often requires ground-up development

FOSS business applications gradually emerging



NTT Comware

Conducts development to improve GNU/Linux reliabilityCluster solution for non-stop GNU/Linux servers

Ultra Monkey load balancing software (FOSS)Linux Virtual Server (LVS) load balancing softwareLdirectord for monitoring failures in real servers and removing failed serversHeartbeat for monitoring operating status of each server

Member of OSDLContributes to development of Carrier Grade Linux (CGL) for telecommunications carriers

GNU/Linux consulting, system building and operationsNew markets for GNU/Linux deployment

Apartment entry management system using fingerprint authentication and L-Box embedded GNU/Linux serverPrototype university business system for Waseda University

Developed using GNU/Linux, PostgreSQL, PHPAvailable free of charge to universities across Japan



Nihon Unisys

Provides high reliability comparable to UnixAlicia Ø Advanced Linux Crash-dump Interactive Analyzer Ø

Sponsored by Japan FOSS Forum’s Development Infrastructure Working Group

Provides environment for kernel crash analysisImproves reliability of enterprise GNU/Linux

Integrates crash and lcrash tools for analyzing Linux kernel crash dumps, enabling faster dump editing

Migrated Unix-based database to GNU/Linux for leading travel agency

Provide applications using FOSSCFIVE learning management system for higher education institutions

Co-developed with University of Tokyo’s Information Technology Center (ITC)

Learning management system (LMS) for e-learning; released as FOSS

Implemented using Java, Tomcat4, PostgreSQL, OpenLDAP



FOSS Deploymentfor Large-Scale Systems

TSUTAYA onlineMigrated to GNU/Linux from Sun Solaris OS, resulting in one-quarter to one-fifth savings in deployment costsLarge-scale system designed for 10 million users

Lawson “Loppi” kiosk terminalsKiosk terminals installed at convenience stores

Rare success among convenience store kiosk ventures

Uses GNU/Linux server to control kiosk terminal used for ticket reservations, etc.

Back-end servers installed behind each storeDeployed in over 7,600 stores

System easy to manage due to remote operationIBM Japan responsible for maintenance



Distributors

Sell packaged FOSSManage software to sell on ease of use

Provide automatic software updatesDevelop proprietary installers or package managersLocalize (Japanese language versions)

Bundle with commercial software and commercial resources (fonts, etc.)Provide support services

Critical to decide on method of creating added value to drive sales

Packing software onto CD-ROM translated into value when networks were not as accessible

Walnut Creek’s distribution businessRoots of GNU/Linux distributions



Ebb and Flow of Distributors

DistroWatch.comPortal site for GNU/Linux distributions

Also lists select BSDs, etc.

List of worldwide distributions

Over 300 GNU/Linux distributions listed

Provides various information about distributions

Reviews, new release information



In-House FOSS Business

Develop own software and release as FOSSEffectively used by enterprise consortiaPrimarily business application software for specific applications

Case studiesNetwork Applied Communication Laboratory (NaCl)

Developer of Japan Medical Association ORCA system; deploying ORCA as FOSS receipt computer

Ten Art-niGaragardoa table reservation system for restaurant industry

ForeOneFirstFrancine Web POS system



Network Applied Communication Laboratory (NaCl)

Developer of software for ORCA standard receipt computer from Japan Medical Association (JMA)

Commissioned by JMA

Drives down cost of expensive receipt computers, by deploying PC servers and FOSS

JMA proprietary license, resembling GPL

Adoption rateStarted as confidential project in 2000

Official launch (March 2002)

Deployed at 1,279 facilities (as of July 2005)

ORCA business over 100 system integrators authorized by JMA for ORCA deployment and contract maintenance

Some system integrators have released modified software

Japan MedicalAssociation

use,improve

NaCl

ORCA

Certifiedpartner

Certifiedpartner

Certifiedpartner

order

development

Hos-pital

Hos-pital

Hos-pital

Hos-pital

Hos-pital

Hos-pital

Hos-pital

deploy,maintenance

NetworkCenter



Ten Art-ni Corporation

Corporate member of OSCAR AllianceVoluntary non-profit association to promote enterprise FOSS deploymentGaragardoa table reservation system for restaurant industry

LAPP system (Linux, Apache, PostgreSQL, PHP)Released as FOSSCo-developed by Ten Art-ni and New Tokyo restaurant chain

Cerveza order issuance and acceptance system for restaurant industry

Co-developed by Ten Art-ni and New TokyoGNU/Linux, Apache, Java servlet, Oracle systemUsed by several corporations



ForeOneFirst Co., Ltd.

Francine Web POS systemSought to increase sales as proprietary product, but product recognition was low

Released by OSCAR Alliance (June 2003)

OSCAR proprietary license, resembling GPL

Main functions: cash register, sales history, product ordering, inventory management, store management, employee attendance management

Francine businessPromote sales through association with OSCAR Alliance brand

ForeOneFirst ultimately acts as core provider of deployment, customization and support services

ForeOneFirst also provides support services for other vendors in OSCAR Alliance

ForeOne-First

NPO OSCARAlliance

Francine

memberVendor

FOSSsupport

Cus-tomer Ø¿

Cus-tomer

Cus-tomer

memberVendor

Cus-tomer

Cus-tomer

Cus-tomer Ø¿Cus-tomer

deploy,operation,maintenance

deploy,operation,maintenance

provideFOSS

use,improve



FOSS Specialization

Conduct business focusing on specific FOSSKey factors to success: Ability to hire top engineers from community and collaborate with communityRequires engineers with advanced skills

Case studiesVA Linux Systems Japan (Linux kernel, NFS)Zend Japan (PHP)SRA (PostgreSQL)SoftAgency (MySQL)Sendmail



SRA, Inc. Japan

Advantages of independent software development vendor

Not tied to specific productStaffed by many Unix engineers

Conducts business focusing on PostgreSQLCollaborates with FOSS community and major developers in Japan and abroadProvides expertise

Sells Windows native versionProvides tuning and other support services for deployment and operationsImplements exams for database certificationRaises profile by publishing articles in magazines and books



Zend Japan, Ltd.

Conducts business focused on PHPParent company Zend Technologies founded by main developers of PHPCapitalizes on technical expertise as original developers of PHP

Offers PHP development environmentProvides expertise to improve PHP performance

Script compilerCaching feature

Web systems development using PHPEnterprise systems

SugarCRM on Zend: Customer management applicationSystem pairing J2EE with PHP applications



Good-Day Inc.

Contract systems development with aggressive support for FOSS

Also offers advanced technical services such as tuning for GNU/Linux and PostgreSQL-based systemsStaffed by FOSS engineers to provide synergistic benefits

Leading example as FOSS specialist vendorEmploys Sylpheed developer and provides financial backingDirectly sponsors FOSS development for Japanese environmentsWorks with OpenOffice.orgSales and support of laptops loaded with FOSS desktop software



Package Software Business

Proprietary package software made to support GNU/Linux

Primarily implemented for server software to expand platform supportClear-cut business model for proven software

Product must be clearly differentiatedWithout differentiation, users will choose FOSS

Barriers for new software

Requires strong degree of differentiation in terms of performance, features, ease of use, development environment, etc.

Market established for network, server and storage management

Opportunities in business knowledge field (accounting, legislation, administration, etc.)

Not a strength of FOSS developersOpportunity to improve interface for beginner users

Business opportunity to develop GUI for general users, that works with existing FOSS, etc.



Server Sales and Embedded Business

Server salesSales of own hardware pre-installed with GNU/Linux

Toshiba, Dell, Sun MicrosystemsHouse-brand PCsOutside of servers, other applications include factory automation, specialized applications, etc.

Embedded businessInformation appliances

Media servers, hard disk recorders, PDAs, mobile phones, etc.Consumer Electronics Linux Forum (CELF)

Established: July 2003Association to study GNU/Linux standards for consumer electronics from Matsushita (Panasonic), Sony, etc.

Industrial equipmentBRAINS: Web camera server system based on NetBSD



Embedded Development Tools Business

Customization of FOSS for embedded systems; sales of development tools and provision of support services

Case studiesMontaVista Software Japan: MontaVista LinuxLineo Solutions: uClinuxAXE: axLinux, ShikigamiRed Hat: Contract porting of GNU/Linux

FOSS embedded business offers excellent opportunities for contract development

Since platform is not a differentiating factor, CE manufacturers are pushing for common platform using FOSSSteady stream of business for contract development dependent on hardware

Soum Corporation: Employs leading developers of NetBSD



Proprietary Business Use

Deliver own services at low cost by deploying GNU/Linux for servers that demand high reliability

amazon.comGoogle.comRakutenLiveDoor

FOSS also useful for rapid deployment of own services and for modification

Take full advantage of modifiable source codeEmploy many FOSS engineers



Amazon.com, Google

Amazon.comUses multiple HP GNU/Linux machines

Distributed Web server cachingSupports over 20 million product inquiries per hour

GoogleFault-tolerant system based on low cost IA servers

Proprietary automatic failure detection system based on Red Hat LinuxDistributed system to account for high rate of hardware failure and distribution of data across multiple hard disks

Google File System for filing dataMap/Reduce Framework for I/O schedulingGlobal Work Queue for scheduling queries



Rakuten, LiveDoor

RakutenDeploys FOSS extensively for its virtual mallsWebLogic and Oracle licenses are costlyFOSS deployed based on balance of service and cost

Primarily uses PHP and MySQL, also PostgreSQL and TomcatSoftware lacks some functions of commercial software

Issues: Shortage of engineers and test suites, frequent version upgrades

LiveDoorDeploys FOSS extensively for its portal sites

FOSS suited for rapid deployment of services

Strategically involved in GNU/Linux distribution businessPurchased Turbolinux unit from SRA and turned into subsidiarySells Linspire



Education Business

Peripheral business within software industry

Courses, materialsAdequate potential for business as number of users and developers increases“Dekiru” series of how-to books are expected to have constant sales, regardless of software

Certification businessNumerous GNU/Linux certifications already exist

LPIC, RHCE, Turbo-CE, CNA, ComTIA Linux+

Other FOSS certifications have been introducedØ¿¿¿ PostgreSQL

Rapid growth in number of certification courses offered by technical schools, etc.



GNU/Linux and FOSS Certification

Certifications for GNU/Linux engineersAll certifications aimed at network administrators

FOSS certificationsMain purpose: Train engineers to use company’s software

Organization Certification Level Nutrality Cost

NPO LPI JapanLPIC Level 1 Entry

Online exam. Nutral30,000 Yen

LPIC Level 2 Middle 30,000 Yen

TurbolinuxTurbo-CE Entry

Online exam.31,500 Yen

Turbo-CE Pro Middle 21,000 Yen

Red HatRHCT Middle 94,500 Yen

RHCE Expert 47,250 Yen

CompTIA Linux+ Novice Online exam. Nutral 27,825 Yen

Zend Japan Entry Online exam. 33,500 Yen

SRAEntry

Online exam.13,650 Yen

Middle 18,900 Yen

TestingMethod

Vendordepended

Practicalexam.

Vendordepended

Zend PHP Certification

Vendordepended

PostgreSQL CE Silver Vendor

dependedPostgreSQL CE Gold



Training Courses

Two main types of GNU/Linux/FOSS training coursesNetwork administrators

Courses basically designed to complement certification exams such as LPI, etc.

Developers of Web system softwareLAMP/LAPP systems developmentIncreasing number of courses for development of Java application servers

IT schools and major IT vendorsOffer GNU/Linux/FOSS training with other coursesMostly short courses ranging from few days to 1 or 2 weeks

SchoolsTechnical schools: 1-3 year GNU/Linux/FOSS engineer training programsProfessional graduate schools: Development of advanced FOSS engineers



(8) FOSS Government Policy and E-Government

Reasons for FOSS promotion by national and local government

Considering reasons why does government promote FOSS?

Trends in EuropeExamples of Germany (Munich), UK, and France

Trends in the United StatesUS government, state governments (case study in Massachusetts)

Trends in AsiaSituations regarding FOSS in east and southeast Asia

Trends in national and local Japanese governmentFOSS adoption trends and their policies



Reasons for FOSS Promotion by National and Local Government

Reduce costs and vendor lock-inEfforts to make sense in government procurement

Transparency of procurement and securityTransparency of procurement and ensuring of security through the utilizing the nature of FOSS are expected

Fostering the development of domestic industryFOSS critical to promoting national IT Industry

Overcoming the “Digital Divide”FOSS is very efficient with low investment for the introduction of IT



Reduce Costs and Vendor Lock-In

Reduce costsThought to be chief advantage of deploying GNU/Linux

Nearly always true when replacing UnixTCO gap with Windows is small and debatable

Focus turns to maintenance costsMaintenance of middleware and applications; cost of administrative engineers

Purpose of government to spend taxes by generating and providing public services; governments seek to deploy FOSS for improved efficiency

Prevent vendor lock-inImportant from standpoint of fairnessExtreme example of vendor lock-in: “One yen bid”

-> See Vendor Lock-In



Transparency of Procurement and Security

Transparency necessary for government procurementWTO Ministerial Conference

Established Working Group on Transparency in Government Procurement Practices

Ensure transparency and competitiveness of public procurement

IssuesClosed public procurementPrevent government-initiative collusion

Since source code for FOSS is released, using FOSS promises to ensure transparency of procurement

SecurityChina cites need to ensure national security as one reason for promoting FOSSBackdoor mechanisms cannot be hidden with FOSS



Fostering the Development of Domestic Industry

Critical mission of governments to promote domestic IT industry

IT vendors from US dominate almost all of package software marketGovernments dislike dependency on foreign sources for high revenue, core softwareDesire to avoid “black boxing” of infrastructure technology

Strong FOSS emphasis in Europe and AsiaGermany: Home to leading FOSS vendors such as SUSE, KDE, OpenOffice, etc.

China: Announced policy to restrict government procurement to domestic software

Japan: Local governments active in fostering local IT industry

Hokkaido, Okinawa, Gifu, Nagasaki Prefectures

Desire to eliminate subcontractor structure for local IT firms



Overcoming the Digital Divide

Digital divideDigital divide created by gaps in income

Digitization among high-income earnersLow-income earners cannot obtain digital equipment

FOSS effective in promoting digitization among low-income earners

Enables IT advancement with limited fundsAlso critical to provide cheap hardware and affordable education

One of the acute problems facing Southeast Asia and South Asia in particular

Overcoming the digital divide in developing countriesFOSS deployment expected to solve this situation



Trends in Europe

European UnionPromoting FOSS in aspects of lower cost and interoperability

GermanyEntering the next stage of “how to deploy FOSS” from the initial stage of “why FOSS”

Case study: Munich Municipal GovernmentMigrating to 14,000 GNU/Linux PCs over 5 years

UK and FranceUK: going deeply into FOSSFrance: walking its own independent path regarding FOSS



European Union

Main focus on reducing costs of e-government and ensuring interoperability

FOSS ideal from standpoint of both cost and interoperability

Promoting FOSS deploymentEC recommends governments deploy FOSS to reduce e-government costs totaling 6.6 billion Euros (July 2002)Launched portal site to encourage spread of FOSS (December 2003)Established COSPA to promote FOSS deployment (April 2004)EU’s IDA Programme recommends OpenOffice.org format as standard office document format for data exchange within government sector (May 2004)



European Union (cont.)

UNU-MERIT FOSS economic impact study (2006)Estimates value of FOSS at 12 billion EuroExpects FOSS-related economy to account for 4% of GDP in the EUIdentifies FOSS as area in which SMEs find good environmentExpects FOSS-related jobs to counter brain-drain towards USA and to close IT gap between EU and USA



Germany

Germany at forefront of FOSS deploymentDebates focused today “How to deploy FOSS”

Case studiesFederal Ministry of Interior signs comprehensive procurement contract for IBM/SUSE Linux machines (June 2002)

Enables low cost procurement of GNU/Linux servers

Federal Ministry of Interior announces guidelines for FOSS migration (July 2003)

Contains list of replacement software and precautions for FOSS migration, for desktop through to server environments

Actual FOSS migration increasing at local government levelMunich city decides to migrate 14,000 machines to GNU/Linux (May 2003)

Schwäbisch Hall district migrates 400 machines to GNU/Linux (2002-2004)

Audit office of Mecklenburg-Western Pomerania completes migration to GNU/Linux (November 2004)



Case Study: Munich Municipal Government

Munich city council decides to undertake GNU/Linux migration for 14,000 PCs used by 16,000 users (May 2003)

ReasonsTo reduce costs? (Not the biggest reason)

IBM/SUSE proposal: $39.5 millionMicrosoft proposal: $36.6 million, later cut to $23.7 million

Avoid dependency on specific productsPromote market competition

“Soft” migrationGradual migration over 5 yearsReplace with Web-based systemsEmulators also to be used initially (VMWare, etc.)

Aug.D ec. 2002 client survey conductedM ay 28, 2002 m igration to OSS resolved at city councilJun. 2002 Jun. 2004 detailed planning with IBM and SU SEJun. 16, 2004 authorized at city councilJun. 2004 Linux m igration project ̀ LiM ux' beganAug. 2004 project stopped bacause of patent issueSep, 2004 project restartedJan. 2005 trial m igration started



United Kingdom

UK: Leaning toward FOSS from neutral stanceFOSS Policy recommends government procurement of FOSS (July 2002)Office of Government Commerce (OGC) decides to deploy GNU/Linux for large-scale system (April 2003)OGC issues report on FOSS trials (October 2004)

FOSS considered to be realistic desktop alternativeFOSS anticipated to reduce costs; no issues with interoperability



France

France: Independent approach to FOSS deployment

Ministry of Culture and Communications to conclude across-the-board GNU/Linux migration in 2005French government establishes ATICA (now ADEA) to promote FOSS (July 2002)Large-scale deployment of FOSS desktops studied as part of ADELE strategic plan for electronic administration (February 2004)

Calls for GNU/Linux migration on 5-15% of desktops by 2007

Defense Ministry signs 3-year, 7 million Euro contract with consortium of Mandrakesoft and other companies to gain CC-EAL5 certification (September 2004)



Trends in the United States

US governmentAlthough US Federal government has “No Policy”, actual FOSS adoption are increasing

State governmentsFOSS deployment is gaining momentum

Case Study: State of MassachusettsThe policy stated that new application should adhere to open standard and FOSS

They have FOSS-based application to be developed under the new policy

Released in the repository, GOCC (Government Open Code Collaborative)



US Government

“No policy” stance by US federal government:Strong opposition by leading IT vendors

Actual FOSS deployment by US federal government is increasing

FOSS already deployed by 250 government sector institutions (May 2002)Government reports recommend FOSS deploymentPITAC recommends FOSS for supercomputers (September 2000); MITRE recommends FOSS for Defense Department (October 2002)

FOSS inclusion in government procurement standards

Defense Department issues memo containing guidelines for FOSS acquisition (June 2003)CC-EAL3 certification acquired for IBM/SUSE platform (January 2004)



State Governments

Movement to deploy FOSS among some state governments

Oregon, Texas, Hawaii and other states have proposed bills for FOSS deployment, which were rejected or shelvedState of Massachusetts introduces preferential policy toward FOSS (September 2003)

Policy later toned down to consider “all possible alternatives – proprietary, open source, and public sector code sharing – in determining best value solutions” (January 2004)

Austin City implements GNU/Linux and OpenOffice.org trials (December 2003)Start of collaborative project to develop FOSS for e-government, led by Massachusetts State (June 2004)

The Government Open Code CollaborativeCalifornia Performance Review Commission recommends increase in FOSS deployment by California state government (August 2004)



Case Study: State of Massachusetts

Virtual Law OfficeCost-cutting move necessitated by financial difficulties due to drop in state tax revenues

CIO meeting proposes policy to deploy open standards and Free/Open-Source

Migration of legacy systems to GNU/Linux platform

SUSE Linux running in virtual machine on mainframe

Red Hat Linux Advanced Server running on IA server

GOCC (Government Open Code Collaborative) Launched in December 2003

Framework for sharing source code with other state governments

Reduce costs by sharing code with other states

Enables continued collaborative development through community effort

MIT and Harvard provide technical and legal support

11 state agencies involved, from eight states



Trends in Asia

East AsiaChina, Taiwan and KoriaNortheast Asia FOSS Promotion Forum: cooperation between Japan, China and Korea

Southeast AsiaThailand, Malaysia, Philippines and VietnamAsia FOSS Symposium: held twice a year



East Asia:China, Taiwan and South Korea

ChinaNational GNU/Linux strategyGovernment-led development of Chinese versions of GNU/Linux

Red Flag Linux, Yangfan Linux, Qihang Linux

Focus on human resources developmentGNU/Linux “1+1+1” Project

TaiwanMany initiatives to promote FOSS

Hosts many FOSS international conferences

South KoreaFlood of GNU/Linux distributors and ongoing market realignmentSouth Korean government rapidly deploying FOSS



Northeast Asia FOSS Promotion Forum

International collaborative project by Japan, China and South Korea

1st Forum: Beijing, China (April 2004)2nd Forum: Sapporo, Japan (July 2004)3rd Forum: Seoul, Korea (December 2004)

Established joint working groups that engage in activities

Technology Development & Assessment (WG1), Human Resource Development (WG2) and Study on Standardization & Certification (WG3)

Supporting organizations formed in each countryJapan FOSS Promotion Forum created in Japan

Secretariat: Information-Technology Promotion Agency, Japan (IPA)

METI, Ministry of Internal Affairs and Communications (MIC), and Japan Information Technology Services Industry Association (JISA) involved as observers



Southeast Asia:Thailand and Malaysia

ThailandLeading FOSS nation in Southeast AsiaNational Electronics and Computer Technology Center (NECTEC): Core organization for promoting FOSS

Sales of 100,000 GNU/Linux PCs through ICT PC Project (now called People’s PC Project)Development of GNU/Linux Thai Language Extension (TLE), etc.

MalaysiaPromoting FOSS through national government and various government organizations

MIMOS (The Malaysian Institute of Microelectronic System)Established the Asian Open Source Center (ASIAOSC)

MAMPU (Malaysian Administrative Modernization and Management Planning Unit)

Announced Public Sector Open Source Master Plan (July 2004)



Southeast Asia:Philippines and Vietnam

Philippines

Anti-piracy a driving factor for promoting FOSSRampant software piracy (reportedly over 60% of all software)

FOSS promoted as part of e-Philippine program

VietnamRapid acceleration in FOSS promotion, sparked by Asia FOSS SymposiumApproved master plan: Applying and Developing Open Source Software in Vietnam for the 2004-2008 Period (March 2004)

FOSS training coursesConferences on FOSS migration, etc.



Asia Open Source Software Symposium

Jointly sponsored by Center of International Cooperation for Computerization (CICC), Japan and organizations in host nations

HistoryFirst Symposium: Phuket, Thailand (March 2003)

2nd Symposium: Singapore (November 2003)

3rd Symposium: Hanoi, Vietnam (March 2004)

4th Symposium: Taipei, Taiwan (September 2004)

5th Symposium: Beijing, China (March 2005)

6th Symposium: Colombo, SriLanka (September 2005)

Features of Asia FOSS SymposiumImpact on host nationDiscuss topics related to FOSS

Share information about status of FOSS promotion in each region

Cooperate on FOSS promotion in Asia



Trends in National and Local Japanese Government

FOSS Policies by Ministry of Economy, Trade and

IndustrySurvey on FOSS usage and FOSS deployment guidelinesFOSS feasibility trialsIPA sponsorship of FOSS development by private sector

FOSS policies in e-government by Ministry of Internal Affairs and Communication (MIC)

Study group concerning “Secure OS”Collaborative Outsourcing / e-Municipality Promotion Strategy

Trends in FOSS deployment at local government levelPart of regional development policyAvoidance of vendor lock-in and emphasis on cost

Case studies: FOSS procurement by local governmentsCase study: Nagasaki prefecture and Hokkaido government



FOSS Policies by Ministry of Economy, Trade and Industry

Policies to promote Japanese IT service industry and software industry

Current massive trade deficit in softwareLoss of technical capabilities from erosion of Japanese infrastructure softwareShortage of human resources development for Japanese engineers, due to drastic increase of FOSS market

FOSS usage survey and guidelines for studying FOSS deployment

Wide overview of major FOSS, FOSS licensing and business models using FOSS

IPA’s Infrastructure Building Program for Open Source SoftwareFunds development related to perceived areas of FOSS weakness

Enterprise technologies

Embedded field

Desktops

Issues unique to Japanese language (fonts, printing, character encoding)



FOSS Policies in E-Government by Ministry of Internal Affairs and Communications

Study Group Concerning Secure OSReport examining security requirements for procurement of FOSS operating systems by e-government

Procurement of specified operating systems is “inadequate”; procurement should be decided based on overall assessment of functions and quality

FOSS comparable to commercial software for functionality and performance

Precautions for government procurement of FOSS operating systems

Vital to include support contract

Additional software may be required if higher level of security is needed

Collaborative outsourcing strategy to promote e-municipalityCollaborative development of common systems for e-municipalities

Systems for online application and filing, accounting, payroll, etc.

Employs FOSS-like development methodSeveral prefectures enlisted to develop different e-municipality functions

Source code released without charge; local governments free to modify or use software



Trends in FOSS Deployment by Local Governments

FOSS deployment for e-municipality platformsFOSS operating environments using Java application packages

Urayasu CityPortal sites for local residents

Yamanashi, Kagawa, Okinawa Prefectures, etc.Release of software developed under MIC’s collaborative outsourcing project

Hokkaido, Shizuoka Prefecture

Deployment for internal systemsFOSS servers widely deployed as Web servers, mail servers, etc.FOSS deployment for desktops

Sumoto CityGNU/Linux servers deployed as gateways to local government WANs

Deployed by 800 local governments



Reasons for FOSS Deployment by Local Governments

Systems procurement by local governments and issues for local IT firms

Ever-increasing concentration on centralized vendors due to too much emphasis on established suppliers

FOSS deployment expected to enable participation in systems procurement by small- and medium-sized local vendors that lack technology for specific products, leading to promotion of local industry

Tendency to emphasize established suppliers, due to difficulty of separating systems design, development and operations

Difficulty in judging costs

Regular rotation of IT systems personnel a problemRegular personnel transfers leads to lack of specialized IT knowledge

Switch to long-term assignments for personnel with highly specialized knowledge

Implement resident services and practice information disclosure

Deploy systems according to needs of each situation, without being limited to existing systems

Transparency of deployment process ensured



Systems Procurement by Local Government and Issues for Local IT Firms

Situation for local IT firms

LGs may limit bidding to established suppliersSpecifications for a public tender are left up to vendorsBlanket contracts that cover from defining requirements to systems operationsAdditional development is automatically awarded to the same vendor

Prevent LGs from directly awarding contracts to local IT vendorsLGs are forced to accept expensive systems that suit the needs of vendors

Local IT firms are subcontractors to major vendorsThe subcontractor role tends to prevent local IT firms from developing the skill set needed for project managementVenture firms are not even qualified to participate in biddingSubcontractors tend to be difficult to keep resources to learn new technology

Tiered subcontractor structure is accelerating due to the shortcomings of local IT vendorsThe issue is compounded by the growing scale and the increasing use of shared e-municipality systems

Situation for local governments (LGs)

Vicious circle

Created by M RI based on m aterials of Nagasaki Prefecture



Case Study: Nagasaki Prefecture

Development Spec.(Requirements, Basic design)

Prefecture staff

Test Specification

Local firms

A systemDevelopment Spec.

Test Spec.

B systemDevelopment Spec.

Test Spec.

C systemDevelopment Spec.

Test Spec.

Bidding &Order

Development

Testing

Divide into bidding unit (N)

Development

Testing

Development

Testing

Local firms (2*N)

Systems Procurement by Nagasaki Prefecture

✗ System development is separated by function✗ Specification requires the use of FOSS

Ø Benefits of Using Detailed Specifications Separated by Function ØTasks are narrowly defined, enabling local firms to participate in bidding based on own capabilities and areas of specializationLocal firms with limited extra resources can get involved, due to availability of detailed specifications and awarding of additional contracts if specification changesMany local firms can participate, since contracts are separated into small sizes

Ø Benefits of Specifying FOSS ØSpecifying FOSS enables equal participation by local firms that do not possess their own software packagesSpecifying FOSS enables contracted firms to use FOSS without worry

Created by M RI based on m aterials of Nagasaki Prefecture

Bidding & Order



Case Study: Hokkaido Government

Activities of FOSS CommunitiesØ HOSS: Hokkaido Open Source and Security (NPO)Ø e-Municipal project / Study Group on Open SourceØ Local User Communities - DOLUG : Hokkaido Linux User Group - PostgreSQL User Group Hokkaido Branch - Zope User Group Hokkaido Branch - NoBUG: Hokkaido *BSD User Group

Commitment from the UniversityØ Hokkaido Univ.

(Open System Engineering Course)Ø Industry-University collaboration project:

OpenSOAP

Support from local governmentsØ establishment of FOSS venture firm (Technoface)Ø Hokkaido local govt.

“declaration of departure from Windows”Ø Ebetsu City “Branding Dictionary” projectØ Large scale GNU/Linux adoption to

Hokkaido high-school net

FOSS related firms

Ø Technoface: SIer based on OpenSOAPØ IP telecom : IPT-Linux and remote surveillance serviceØ Cyber blue : Development of Open Source GIS



Education and FOSS

IT use in educational settingsFundamental idea to use IT equipment in the field of education

Situation in Japanese schoolsJapanese public schools suffer from a lack of funding to manage IT equipmentRaising expectations for utilizing FOSS

Aim of software vendorsMotivated by a desire to invest in students as future users

Case Study: FOSS trials in educationSome trial projects to use FOSS in educational field have been conducted



IT Use in Educational Settings

Area of emphasis under e-Japan strategyAdvancement of Human Resource Development and the Promotion of Education and Learninge-Japan Priority Policy Program 2004

Digitization of school educationImprove IT environment in schools

By FY2005, provide 1:1 ratio of PCs to students in com puter labs and 5.4 ratio of students to each educational PC in regular classroom s

IT use in education not information literacy education

Use of IT in education, not education of IT itselfExcept for “Information Study” curriculum in upper secondary schools

IT use aimed at boosting achievement of fundamental goals of education



Situation in Japanese Schools

Inadequate funds to deploy, administer and operate IT equipmentAdministration of equipment

Administration of equipment frequently left up to select group of knowledgeable teachers

Imposes extra workload on teachers

Volunteer staffs from PTA committees or alumni sometimes build systemsInadequate safeguards for security and privacy, despite sensitivity toward these issues

Instruction using IT equipment

Various experimental programs are being triedImplementation varies widely depending on school board, school and teacherActive use of multimedia content

Reaction to FOSS from students

Able to quickly take in new ideasSome teachers commented that they learned new things from students



Aim of Software Vendors

Students and children seen as future usersType of enclosure strategyReasons for academic pricing

Pretext: For cash-strapped studentsReal reason: Capture users at early stage

Issues with Information Study textbooksExamples used to illustrate how applications work

Pros and cons of describing specific applicationsMeaningful to teach operating methods specific to applications?Is it fair to teach students about specific applications?



Case Study: FOSS Trials in Education

IPA-sponsored field tests of FOSS desktops in educational institutions (2004)

GNU/Linux desktops deployed at eight elementary and lower secondary schools in Tsukuba City and Gifu Prefecture

300 machines deployed and used by 3,000 students

Knoppix trialsKnoppix used by 800 students at eight schools across Japan (elementary/upper secondary schools, one technical school and several universities)

Norway’s Skolelinux project (2003)Skolelinux deployed and evaluated at four elementary and lower secondary schools in Norway

http://www.skolelinux.org/portal/documentation/reports/

UK’s Open Source Software in Schools project (2004)FOSS deployed and evaluated at 15 elementary and lower secondary schools in UK

http://www.egovmonitor.com/node/907



(9) Feature of FOSS

PerformanceHow is performance of FOSS products?

CostIs it possible to reduce cost by FOSS?

Vendor lock-inWhat is vendor lock-in?How to avoid vendor lock-in by adopting FOSS?

SecurityIs the security of FOSS product high or low?

Educational benefitsWhat is educational efficiency from FOSS?

SustainabilityImportance of sustainable software development



Performance

Innovative and fast developmentFOSS is suited for innovative developmentDeveloping speed of active FOSS project is amazingly high

FOSS performance testing in JapanAs a matter of fact, how is the performance of FOSS?A series of performance evaluations on FOSS products were conducted by a work group of Japan FOSS promotion forum



Innovative and Rapid Development

Reasons for innovative developmentMotivation of FOSS developers

Create software that developers wantCreate software unlike any other

Localization tends to be more easily implementedLocalization undertaken by actual users that desire localization

Rapid pace of developmentProjects fundamentally driven by enthusiasm of developers

Although development sometimes takes place through business

Desire to quickly develop software that developers wantLarge projects involve worldwide developer base

Round-the-clock development



FOSS Performance Testing in Japan

Performance evaluations implemented by working group of Japan FOSS Promotion Forum

Results of Development Infrastructure Working GroupDBMS benchmark evaluation using OSDL DBT-1

Benefits of tuning confirmed

PostgreSQL evaluation using OSDL DBT-3JBFOSS performance and reliability evaluation using SPECjAppServer2004

Performance of WebLogic surpasses JBFOSS

Evaluation and bottleneck analysis of Linux kernel using LKSTDevelopment of crash analysis tools and evaluation of FOSS performance and reliability

Evaluation of Java application layerEvaluation of database and OS layers

Members of Development Infrastructure Working GroupHitachi, SRA, NTT Data, NS Solutions, Sumisho Computer Systems, NRI, Miracle Linux, Uniadex, NTT Comware, Nihon Unisys



Cost

Myth of low costFOSS is not almighty to reduce costs

Evaluating total cost of ownership (TCO)Cost evaluation strongly depends on assessor's idea

Cost reduction factorsWhat items can be candidate for cost reduction factors by FOSS adoption?

Cost disadvantage of FOSSWhat are disadvantage points in FOSS adoption?



Myth of Low Cost

Presumed to be chief advantage of GNU/Linux deployment

Nearly always true when replacing UnixTsutaya Online: 1/4 to 1/5 cost reductionAmazon.com: Saved $17 million

TCO gap with Windows is small and debatableInadequate experience and lack of thorough discussion for meaningful comparison of desktops costs

Focus ends up being on maintenance costsMaintenance of middleware and applicationsCost of administrative engineersOnce technology is created to easily manage multiple desktops

Desktop tug-of-war between Windows and GNU/Linux



Evaluating Total Cost of Ownership

TCO (Total Cost of Ownership)Total of all costs required to maintain, administer, and adequately operate systems

Elements contributing to TCODeployment costs, operating costs, training costs, etc.

Room to calculate TCO as desired, by changing standpoint of evaluation

Microsoft’s Get the Facts campaign: How are the facts represented?

Various assumptions are suspectProper evaluation of TCO requires concrete assumptions about how system will be usedResults will depend on the assumptions made



Cost Reduction Factors

Initial deployment costsCost of acquiring distribution (only one copy needed)Licensing costs

If proprietary software is required, only license for that software must be purchased

Upgrade costsUsually very lowFrequently expensive for proprietary software

Hardware costsRuns adequately on older hardware for certain applications

Systems can be tuned to adequately run on low spec PCs



Cost Disadvantages of FOSS

Frequently identified cost disadvantages

Training costsUsers are generally conservative and resist migrating from a familiar environment to a new environmentLack of textbooks and reference material for mastering FOSS

Support costsHigh cost of support due to limited supply of FOSS engineers

Costs are going to decrease as FOSS becomes more common

Systems modification costsServers may need to be modified

Some systems only designed to work with specific clientsFOSS clients may not work within such systems



Vendor Lock-In

What is vendor lock-inAn environment or situation that locks in users to products from a specific vendorWhat are the problems with vendor lock-in?

Examples of lock-in business strategiesVendor lock-in is found not only in IT market

Separation of Interface and ImplementationPromoting fair competition among implementations from each vendor, in conformity with standard interfaces

FOSS and open standardsReasons why fair competition requires more than specification standards



What is Vendor Lock-In

Vendor lock-inLocks in user environment to specific vendor’s productsType of customer retention strategyCan lead to endless vicious circle of biased procurement

Forces users into regular version upgradesMonopoly can invite lower quality and higher costs

Need to maintain compatibility with archival assetsEliminates participation by other vendors through use of closed specifications

Used to be an excellent business modelUsers: Buy long-term support and peace of mindVendors: Stable revenue and growth of market share



Examples of Lock-In Business Strategies

Rewards programsLeading examples

Credit card companiesFrequent flier programsMail order companies and mass merchandisers

Everyday examplesFrequent buyer cards from retailersRestaurant coupons

Car dealersPreferential trade-in programs for car brands from same manufacturerFrequent model changes

Mobile phone carriers and ISPsFrequently introduce new models and servicesUsers do not wish to change mobile numbers and e-mail addresses

Introduction of mobile number portability may reduce lock-in advantage

MNP: Mobile Number Portability

Worst case of vendor lock-in“One yen bid”

Bid on first year at low costGain highly profitable private contracts in after the second year



Separation of Interface and Implementation

Separate from implementation to achieve competition1. Derive necessary functions and separate into modules.

2. For each module, separate the interface and implementation.

3. Define the interface and establish it as a standard.

Separation of implementation ideally results in fair competition

From de facto standards to open standardsConventional de facto standards

Implementation also treated as part of standard

Open standardsStandards formulation process also handled openlyImplementation left to each vendorInterface is specified to a standard, and implementation is interchangeable



Free/Open-Source Software and Open Standards

Avoid vendor lock-inMake specifications open to maintain competition

Open specifications alone are insufficient

→ Also need to release source codeEnables other vendors to participate in system upgrades

User has upper hand for price negotiations

Why open standards alone are inadequateVendors will always emerge to seek differentiation through proprietary meansExample of HTML

Standard specifications decided by W3C

Browser war: Browser incompatibilities due to proprietary tag extensions

Example of tying applications into OSProprietary performance enhancements using unpublished APIs

Vendor gave itself unique advantage as OS developer



Security

Security of SoftwareWhat is secure software? From whom do we have to protect our software?

Is FOSS really more secure?Opinions from two sides: “More Secure” vs “Less Secure”

“Many eyes” of developersThe reason why FOSS is more secure

Naked implementationThe reason why FOSS is less secure

FOSS Security ToolsThere are many security tools released as FOSS



Security of Software

Security is the ability to keep software operating as expected

Obstacles to securityOutside factors

Unauthorized access: Unexpected manipulation by third party

Interception and falsification: Incidents relating to transmission path for privileged data

Interception: Unexpected leakage of dataFalsification: Transmission of insidiously modified data

Internal factorsSoftware defect: Unexpected operation due to fault in software

Vulnerability: Fault or specification issue that could be used by third party to take over system or leak data, etc.

Raises the issueCan FOSS improve security against these threats?



Is FOSS Really More Secure?

Widely divergent viewsFOSS is more secure due to transparency and active developer baseProprietary software from major vendors affords greater chance of ensuring security

Reality of the situationServers

Unauthorized access and security accidents occur on all platforms including Unix, GNU/Linux and Windows

DesktopsViruses and worms targeted at FOSS are extremely rare

Low probability due to small number of FOSS desktops to begin with?

Depends on system design principles; operating systems designed for convenience are comparatively more vulnerable



“Many Eyes” of Developers

Possible reasons for greater security of FOSSRapid response to defects

Continuous round-the-clock development by worldwide developer baseRelatively fast response when critical vulnerabilities or security holes are identifiedApplies to actively developed FOSS projects

“Trojan horse” measuresDifficult to slip unauthorized code into source code circulated as FOSS

Based on notion of improved security through vigilance of many developers



Naked Implementation

Reasons FOSS is thought to reduce securityReleasing source code provides crackers with enough information to do harmEasier to find errors or security holes in implementation

Counter-argument: Release of source code enables immediate response if security holes are discovered

Concern about structure for security measures by software provider

Major vendors are putting resources into security measuresLack of trust in volunteer development of FOSS projects

Risk is higher for less active FOSS projects



FOSS Security Tools

GNU Privacy Guard (GPG)PGP encryption tool from GNUMany MUAs work with GPG

Snort, CodeSeekerFOSS Intrusion Detection Systems (IDS)

OpenSSHFOSS implementation of Secure Shell (SSH) protocolCommonly used today in place of Telnet and remote shell (rsh)

OpenSSL, GNU TLSFOSS implementations of SSL (Secure Socket Layer) protocol

Other tools: OpenVPN, Tripwire (system integrity check tool), etc.



Educational Benefits

Source code as an exampleLearn from the precedence of released source codeLeading source code is equivalent to an excellent text book

Using a debugger to verify that source code runsImportant issue is that the source code is really runnable

Low cost of development environments and resources

Low entry levels to start learning

Educational benefits of communitiesWe have much from communities



Source Code as an Example

Study concrete examplesSoftware design methodologiesProgramming techniques

Actual programming samplesLearn step-by-step how a program runsLook up similar code

Source code as a textbookPublications relating to FOSS such as Code Reading and Lions' Commentary on UNIX 6th Edition with Source Code are published



Using a Debugger to Verify that Source Code Runs

FOSS makes it possible to:Obtain source codeModify source codeCheck source code

Source code that actually runs

Important to verify using debugger

Verify operating logicLearn through practical experience

Enables study using real code



Low Cost Development Environment and Development Resources

Proprietary software involves tall barriers to participation

Purchase of development toolsPurchase of development informationFee-based training

Development resources comparatively lower cost for FOSS

Distributions include development toolsInformation available onlineTraining largely fee-based

Risks for software technology acquisition under FOSS

Time and effortAbility to control risks yourself is where FOSS excels in



Educational Benefits of Communities

Approach that users can learn how to use software from the community without learning by users themselves

User communities as cooperative organizationsAsk questions through mailing lists and message boards

Rules of communitiesGive-and-take of informationObserve netiquette

Prevent flame wars from occurring

Development communities must also make an effort to provide accessible information



Sustainability

Necessity of sustainable software developmentWhy sustainability of software development is so important?The key is adaptation to various kind of platforms

Realizing sustainable software developmentWhy FOSS enables us to realize sustainable software development?

Scientific progress and advancement of softwareSimilarities between scientific progress and advancement of software

For the advancement of softwareTo produce better software products



Necessity of Sustainable Software Development

Sustainable software development is necessary as long as users wish to use software

Defects in software discovered on daily basisSecurity holes cause problems for others

OK to end development of tried-and-tested software*?*Tried-and-tested software: Software in which almost all bugs have been worked out

Are there any issues with suspending software development?

→ Yes, there areReasons

Changes in surrounding environment including OS, dependent libraries, etc.

Software must be adapted to support to these changes

FOSS capable of being adapted to diverse platforms



Realizing Sustainable Software Development

For proprietary softwareRisk of software development ending for some reason



Scientific Progress and Advancement of Software

Advancement of software and scientific progress fundamentally similar

Compare the following points

Advancement of software (under FOSS)Implementation shared and extended through release of source codeSource code must run properly. Avoid reinventing the wheel

Scientific progressKnowledge shared and expanded through publication of papersTest theory through use of corroborative experimentsPointless to conduct the same research afterwards



For the Advancement of Software

Freedom of FOSSThe freedom to run the program, for any purposeThe freedom to study how the program works, and adapt it to your needsThe freedom to redistribute copies so you can help your neighborThe freedom to improve the program, and release your improvements to the public, so that the whole community benefits

Above four kinds of freedoms prescribed by FSF’s The Free Software Definition

Copyleft is abstract expression of four kinds of freedomGPL expresses four kinds of freedom as concrete license

Leads to implementation of sustainable software development

Should lead to advancement of software



(10) Issues with FOSS

Lack of applications, human resources and guarantees

Factors lacking in order to utilize FOSS the most effectively

Legal risksWho secures legal risks in using FOSS?

FOSS deployment on desktopsDesktop use is backward in comparison with server use

DiversityWhat makes the issue of software complex?

Standardization trends and localizationStandardization to eliminate diversitiesLocalization to use domestically in each nations



Lack of Applications, Human Resources and Guarantees

Refinement of applicationsQuality is vary widely between FOSS applicationsHope more contribution for further advancement to high-quality FOSS

Underdeveloped FOSS-related market and support concerns

Underdeveloped in comparison to proprietary software marketBringing up support business is also future task

Shortage of human resourcesNeed to absolute increase of FOSS engineers



Refinement of Applications

Characteristics of bazaar development modelQuality not always guaranteed

Separation of development versions and stable versions

Level of refinement varies widelyTried-and-tested applications are highly refined

Emacs, gcc, X Window System, apache, qmail, etc...

Latest applications may have many hidden issues

Refinement of desktop applications in particular needs to improve

Demanding requirements of users (features and ease of operation)Refinement of desktop applications directly linked to assessment by users



Underdeveloped FOSS-Related Market and Support Concerns

Related markets still undevelopedBusiness deployment of FOSS often still at explorative stageTraining and support markets are smallShortage of FOSS books

Information frequently obtained online

User concerns about supportLack of heavy users nearby

Prefer to have person nearby to casually consult with when problems occur

Support business yet to take firm shapeNo established support services

Concern over whether community can actually be relied on

Some demand for comprehensive support from vendors



Shortage of Human Resources

Shortage in absolute number of FOSS engineersShortage of GNU/Linux engineers frequently pointed out

Easy to actually retrain Unix engineers

High labor rates due to lack of supplyConcern that situation will drive up overall costs

Need to secure support personnelNumber of engineers steadily increasingNumbers increasing starting from level below engineers

Government policies to develop advanced, hacker-level FOSS engineers

Public funds to support development of engineersSymposiums, seminars and “Codefests”



Legal Risks

Relationship of FOSS vs copyright and patentsAlso weak points of FOSSPossibilities to bear risks of intellectual property rights infringement

SCO controversySCO sent a challenge to FOSS communities in 2003

Responsibility of warrantyWho has responsibility for defects in FOSS?

License violationsGPL violations occur every day in the world?



Relationship of FOSS versus Copyright and Patents

Many developers participate in FOSS style development

Risk of containing code that infringes on copyright or patents; code can slip in intentionally or through carelessness

Infringement on patent licenseNormal business response

Cross-licensingFinancial resolution (payment of patent royalties)

Both difficult to execute with FOSSRisk of patent royalties being claimed directly against users, or a cease-and-desist order could be issued

Example of guard mechanismsMPLClause to the effect that any withstanding patents applying to source code must be expressly declared by the contributor



SCO Controversy

US-based firm SCO Group issues warning against GNU/Linux users in May 2003

Claim asserts that Unix code was misappropriated for Linux and infringes on intellectual property held by SCO

Claim against Linux kernel Version 2.4 and later; SCO alleges that Linux kernel contains code relating to NUMA, JFS and SMP technologiesSCO alleges that it purchased rights to Unix from Novell

SCO sues IBM for damages, claiming that IBM misappropriated code from AIX (March 2003)

Surrounding response73% of GNU/Linux programmers say that SCO claims have no meritSCO controversy barely receives attention today in 2005



Responsibility of Warranty

Is no one responsible for FOSS?Fundamentally provided “as is” and with “no warranty”Proprietary software: Warranty conditions and exemptions

Does not mean that users can rest easy

Japanese Consumer Contract ActSection 5, Article 8: “Clauses which totally excludes a business from liability to compensate the damage to a consumer caused by such defect” are null and void under a contract for valueSome experts consider exclusion of any warranty unenforceable under Japanese law

Situation can also be viewed as business opportunitySupport businessProvide insurance and warranties, legal risk management, etc.

Open Source Risk Management (US firm)



License Violations

Incidents involving license violationsPrinter driver

Binaries distributed free of charge contained FOSS code, but software was not released as FOSSManufacturer issued apology on Web site and issued replacement software with revised license

DivX Converter contained misappropriated XviD code (FOSS)PornView incident, in which original copyright notices from GImageView were stripped

FSF responds to GPL violationsFSF manages copyrights for Free SoftwareBy transferring copyright to FSF, legal authority and responsibility is centrally managed by FSF

By transferring copyright, FSF has authority to take action against violations



FOSS Deployment on Desktops

Trends in FOSS desktop marketWhat is the trends of software market regarding FOSS desktop terminal use?

Phases for spread of FOSS desktop deploymentWho can use FOSS desktop?

Barriers to spread of FOSS desktopsWhat prevents from spreading FOSS desktop use?

Examples of issues with Japanese language environments

Font inadequacies and confusion over Japanese character encoding prevent from spreading FOSS desktops in Japan



Trends in FOSS Desktop Market

Was 2004 “year one” for spread of FOSS desktops in Japan?

FOSS desktops begin drawing interest in wake of FOSS deployment for servers, backbone systems and embedded applicationsOSDL launches new working group (January 2004)

Desktop Linux Working Group

Several trials for FOSS desktops conductedAIST’s FOSS desktop strategyFOSS trials for schools (IPA and CEC)

Desktop GNU/Linux distributionsJava Desktop SystemTurbolinux 10 DesktopNovell Ximian Desktop 2 (SUSE LINUX)



Phases for Spread of FOSS Desktop Deployment

IT engineers, high-end users

Routine task workersCall centersCounter tasksOther routine work, etc.

Management, sales, general office workers, etc.Issues

Improve interoperability of various electronic data (eliminate diversity)Eliminate diversity in how applications are operated

Reference: Decrem, Bart. Desktop Linux Technology & Market Overview. Open Source Applications Foundation, July 2003



Barriers to Spread of FOSS Desktops

Issues for spread of FOSS desktopsExpanded environment for FOSS office suites

Actual applications need to be more refinedTemplates, clip art libraries, etc.

Refinement of user interfacesMigrate systems to Web applications

Deep-rooted barriers to spread of FOSS desktops in Japan

Inadequate Japanese language environmentDemanding requirements for document style, customary ruled lines, etc.Problems with applications due to Japanese language support

Application hang-ups from kana-kanji conversion, etc.



Examples of Issues with Japanese Language Environments

Font inadequaciesControversy over copyright infringement by free fonts (2003)No free fonts of adequate qualityHigh cost of developing Japanese fonts

Due to large number of characters

Too many character encodingsFor historical reasons

JIS (ISO2022-JP)Shift JISEUC-JP (Extended Unix Code)Unicode

Other Asian nations also face same problem



Diversity

Driving factors behind diversityWhat is diversity?

Diversity of dataIssues on data format handled by applications

Diversity of platformsIssues on environments where applications work

Diversity of user interfacesIssues on differences between operationality of applications



Driving Factors behind Diversity

What is diversity?Freedom to combine components leads to explosion in number of combinationsDiversity reflects a high degree of freedom, resulting in a trade-off relationship with integrated feel

Also happens in proprietary environments

PC-compatible machinesDiversity of devicesAddressed by device manufacturers

Difficult for minority makers



Diversity of Data

Problems arise during external exchange of electronic documents

Data formats specific to applicationsex. *.doc, *.xls, *.ppt,...

Need to exchange actual content, not formatContent in written documents

Written sentences, style, etc.

Content in spreadsheets, presentationsValues, graphs, diagrams, fonts, etc.

Data dependent on individual applicationDifferent data formats within one category of applications

Ex. Word processorsMS-Word, OpenOffice.org Writer, KOffice, AbiWord, etc...



Diversity of Platforms

Excessive freedom in configuring FOSS platformsRange of variables

Kernel parameters, kernel modulesLibraries, toolsOther software to which applications are dependent on

Reasons for platform diversity (diversity of operating environments)

Differences between distributionsRed Hat, SUSE, Debian, Knoppix, etc...

Differences in versionsIndividual applications continually evolve on their own

Same issues also effect proprietary operating systems

Service pack editions installed or not installed, OS editions, etc.



Diversity of User Interfaces

Diversity of operation directly breeds mistrust of usersParticularly true for desktop deployment

Familiar user interface and operating characteristicsEssential features might be same but other differences exist

Different iconsDifferent menu placement or menu titlesDifferent keyboard shortcutsDifferent error messagesetc...

Users are conservativeRelearning new way of operation considered a waste of previous time and effortAfter going to all that trouble to learn how to operate an application



Standardization Trends and Localization

Standardization of GNU/LinuxThe road to ISO standard

Standardization of documentsTrial to make electronic document interchange smooth

i18n, m17n, l10nDifference among internationalization, multi-lingualization and localization

Examples of internationalized applications



Standardization of GNU/Linux

Seek to absorb platform variationsUnitedLinux consortium’s trial

FSG (Free Standards Group)Non-profit organization dedicated to FOSS-related standardization

Develops standardsDevelops compliance testing tools and tests software for compliance

Linux Standard Base (LSB) ProjectDedicated to application compatibility between authorized distributions

Other related standardsPortable Operating System Interface for Unix (POSIX)Single Unix Specification (SUS) Version 2Filesystem Hierarchy Standard (FHS)



Standardization of Documents

OASIS (Organization for the Advancement of Structured Information Standards)

Study by standardization committeeOpenDocument

Based on XMLOpen standards specificationNo platform dependencyNo application dependencyCandidate for EU’s public document formatUsed in OpenOffice.org 2.0 and KOffice 1.4

Distinct from XML-based document format proposed by Microsoft



I18n, M17n and L10n

“I18n” takes first letter “i” and last letter “n,” omitting 18 lettersi18n (Internationalization)

Provision of framework that simplifies implementation of m17n and l10n in applications

Preparation of fonts, input method, basic libraries, etc.Gettext and message catalogs

m17n (multilingualization)Modification to support multilingual handling

Not just characters and words, but also line breaks, dates, currency units, etc.Concurrent handling of multiple languages separates m17n from l10n

l10n (localization)Enables display in each language



Examples of Internationalized Applications

Necessity of internationalized applications

Owner’s manualsTranslation

Need to display and edit at least two languages

Handling of charactersUnicode

Issues with support for existing character encodings

Number of Unicode-compatible applications is increasing

Implementation still incomplete



(11) Development Tools

We have variety of FOSS development tools

Developers make a new tool for themselvesPlenty of tools for distributed development

Based on FOSS development via the Internet

GUI-based tools are increasing, in addition to conventional CUI-based tools

Development toolsCompilerDebuggerAnalyzer, profilerSource code managementMaintaining compatibilityLocalizationDocumentationIDE based on GUIBug tracking tools



Compiler

Process of building softwareCompile

Source code -> object code

LinkSet of object code -> executable code

File describing the process of building softwareMakefile

gcc, make, ldDe fact standard tools for FOSS development



GCC

gcc (the GNU Compiler Collection)Development started in 1984 by Richard Stallman

Originally stood for GNU C CompilerNow stands for GNU Compiler CollectionIncludes compilers and libraries for C, C++, Objective-C, Fortran, Java and Ada

C++ compiler g++Fortran compiler g77Java compiler gcj

FeaturesWidely used for commercial and non-commercial operating systemsCan also be used as cross-compiler



MakeTo help manage build process

Marking of dependency and processing method

Rules (compile, link, installation, etc.) are described in makefile

Autotools convenient for automatic generation of rules

Build optimizationLooks up time of last update and only executes minimum build needed

PACKAGE = hogehogeSRCS = $(PACKAGE).cOBJS = $(SRCS:.c=.o)

FILES = README Makefile $(HEADS) $(SRCS)VER = `date +%Y%m%d`

CC = gccCFLAGS = -g -O2 -Wall $(DEBUG)CPPFLAGS = -I.

.SUFFIXES:

.SUFFIXES: .o .c .cc .f .p

all: $(PACKAGE)

$(PACKAGE): $(OBJS)$(LD) $(LDFLAGS) $(OBJS) -o $@ $(LDLIBS)

$(OBJS): $(HEADS) Makefile

.c.o:$(CC) $(CFLAGS) $(CPPFLAGS) -c $< -o $@

.cc.o:$(CXX) $(CXXFLAGS) $(CPPFLAGS) -c $< -o $@

.f.o:$(FC) $(FFLAGS) -c $< -o $@

.p.o:$(PC) $(PFLAGS) $(CPPFLAGS) -c $< -o $@

clean:$(RM) $(PACKAGE) $(OBJS)$(RM) core gmon.out *~ #*#

Makefile example



The GNU Linker Ld

Invoked at end of build processLink multiple object files and library or archive filesRelocate dataTie up symbol references

Static linkingCombines all object files and libraries from build into one program

Runs as standalone file but produces large file size

Dynamic linkingOnly designates name of librariesDynamically links to libraries during execution

Small file size

# ld -o a.out /usr/lib/crt1.o \/usr/lib/crti.o hello.o -lc



Debugging Tools

DebuggingProcess of fixing bugs in codingBasically check for:

Are values for variables as expected?Is conditional branch correct?

Possible to insert code to output values at various points, but labor-intensiveUse of debugging tools

Debugging toolsDebuggers, profilers, tracers, etc.



Debuggers

GDB (The GNU Project Debugger)

CUI debuggerFeatures: Set breakpoints, step-by-step execution, etc.

DDD (GNU Data Display Debugger)

GUI front-end for GDB and other CUI debuggers



Debugging Case Study (Part 1)

gscanbusTool for acquiring and displaying data from connected IEEE 1394 device

Problem (bug?)Camera icon displayed as question mark

Camera works properlyDevice type not recognized



Debugging Example (Part 2)

Dig through the source codeFile names

Icon.h and icon.c files are suspect

icon.hInitIcons() and chooseIcon() functions called from outsideChooseIcon() is suspect; chooseIcon() function is for initialization

ChooseIcon() in icon.c fileIcon appears to be switched by rom_info>node_type

void chooseIcon(Rom_info *rom_info, GdkBitmap **xpm_node, GdkBitmap **xpm_node_mask, char **label) {

switch(rom_info->node_type) { case NODE_TYPE_CONF_CAM: case NODE_TYPE_AVC: *xpm_node = xpm_dvcr; *xpm_node_mask = xpm_dvcr_mask; *label = "AV/C Device"; break; case NODE_TYPE_SBP2:




Debugging processRom_info.h and rom_info.c files exist

Use grep to search location where value of node_type is setGet_node_type is suspectProbable solution: Change code to return NODE_TYPE_CONF_CAM when camera you are using (unit_sw_version=0x101) is connected

int get_node_type(Rom_info *rom_info) { char cpu; if (rom_info->unit_spec_id == 0xA02D) { if (rom_info->unit_sw_version == 0x100) { return NODE_TYPE_CONF_CAM; } else if (rom_info->unit_sw_version == 0x10000 || rom_info->unit_sw_version == 0x10001) { return NODE_TYPE_AVC; } } else if (rom_info->unit_spec_id == 0x609E && rom_info->unit_sw_version == 0x10483) { return NODE_TYPE_SBP2; } else { resolv_guid(rom_info->guid_hi, rom_info->guid_lo, &cpu); if (cpu) return NODE_TYPE_CPU; } return NODE_TYPE_UNKNOWN;}




ResultIcon is correctly displayed

Unknown if patch is correct method to fix problem

Fix was possible because of viewable source codePatch may be incorporated into next version by feeding back to community



Analysis Tools

Analyze how program runs dynamically and/or check source code statically

To help bugfix, quality improvement and speed up, etc.

Profilers / memory testing toolsTools to get statistical information on CPU and memory

TracersTools to trace function calls and system calls

Source code analyzing toolsctags, etags, etc.

Create tag information by reviewing source codeTo jump directly to the definition of classes and functions

cflowTo show the invocation tree between functions and functions



Profilers

ProfilingAcquire status of program execution at fixed intervals

Show process and thread statusAlso acquire hardware information such as cache hit ratio

CPU profilersMeasure CPU utilization rates

Memory profilersMeasure memory usage, detect memory leaks



GNU Gprof

Utility for measuring and displaying operating status of program

Number of calls for each function, processing time, etc.Shows bottlenecks to consider for acceleration

Using gprofSpecify -pg option when compilingExecute program normally # gprof executable-file gmon.out

Sample outputfunc1 takes up zero timefunc2 has room for acceleration

% cumulative self self total time seconds seconds calls ms/call ms/call name 100.00 0.40 0.40 80 5.00 5.00 func2 0.00 0.40 0.00 3 0.00 133.33 func1



Memory Testing

MemProfProfiler for memory usageTest for memory leaks

MEMWATCHCUI memory testing tool for CDetection of memory leaks, data corruption, etc.



Tracers

Traces function calls and system calls

CTraceTraces function calls

LtraceTraces function calls for shared libraries

Strace for GNU/Linux and ktrace for *BSDTraces system calls

__libc_start_main(0x080664e0, 1, 0xbffff654, 0x08106ab0, 0x08106af8 <unfinished ...>setlocale(6, "") = "ja_JP.eucJP"bindtextdomain("sylpheed", "/usr/local/share/locale") = "/usr/local/share/locale"bind_textdomain_codeset(0x08106c02, 0x08106ba0, 0xbffff608, 0x08106aca, 0x40608968) = 0x08142870textdomain("sylpheed") = "sylpheed"g_get_current_dir(0x08106c02, 0x08142870, 0xbffff608, 0x08106aca, 0x40608968 <unfinished ...>malloc(4097) = 0x08142890Ø¿¿

Sample output from ltrace



Source Code Management

Manages source code itself, the changing history and so on

Essential tool for team developmentTo make sure when, who, where and how modified?

Reverting back to older revision and/or making branch versions are possible

Inevitability of source code management toolRCS, CVS, subversions and other similar tools were developed in association with changes of development styles from independent development to team and distributed development



Diff, Patch

Patchfile (shown at right)Shows differences between files

Diff and patchdiff

Tool to generate differences

patchTool for applying differences to create revised file

Basic features of RCS, CVS and SubversionUsed by someone other than source code administrator to create bug fixes or add featuresRedundant to send entire modified file

Send only differences to administratorPractice dates back to when transmission speeds were slow, making it necessary to reduce data volume

*** hello.c 2005-06-17 17:46:46.000000000 +0900--- nice.c 2005-06-17 17:47:04.000000000 +0900****************** 3,5 **** int main(void){! printf("Hello World!\n"); return 0;--- 3,5 ---- int main(void){! printf("Nice to meet you!\n"); return 0;



Version Control (SCCS, RCS, CVS, Subversion)

SCCS and RCSVersion control for single files

CVSCreates working copies on client based on contents of server (repository)Enables concurrent editing of same file by multiple persons

SubversionAddresses downsides of CVS (inability to move or delete directories, etc.)

Commands in CVS and Subversion

checkoutCreates working copies from repository

commitSaves to repository

updateUpdates working copies

add/deleteAdds and deletes files

diffShows differences between files

statusShows status of files



Maintaining Compatibility

Differences in platformsDifferences in operating systems

GNU/Linux, *BSD, Unix, Windows, etc.

Differences in librariesOpenGL/Mesa, Xaw, Motif/lesstif, etc.

Differences in versionsSpecifications can change due to version upgrades

Differences in paths

Need arrangement for absorbing these differences

Labor-intensive to implement manuallyDifficult to support platforms not possessed by developers



GNU Autotools

Need more than just source codeNeed tools to compile and execute same program on different platforms (OS or environment)

Improve portability and maintainability

Tools such as autoconf, automake, etc.Autoconf: Used to generate configure scriptsAutomake: Used to generate makefiles

Minor format differences depending on version of Autotools used

Contradicts the very purpose of Autotools?



Localization

English is the universal language of FOSS development

Many developers and users lack English skillsDesire for different language versions so that many others can use software

LocalizationEnables display of character strings such as menus and dialogs in different languagesDevelopers do not need to be versed in each languageTranslators do not need to understand source code

Localization tools do not support internationalization

Multilingual handling, line breaks, etc.See I18n, M17n and L10n



GNU gettext

Implements localization without major changes to existing source code

Prepare a message catalogSurround the internationalized character string with N_()

Ex. Change char *str = "Hi"; to char *str = N_("Hi");

Surround variables that call on the string with _()Ex. Change printf("%s\n", str); to printf("% s\n", _(str));

Example of message catalogReplaces “Hi” (msgid) with “Guten Tag” (msgstr)

msgid "Hi"msgstr "Guten Tag"

msgid "Add"msgstr "Hinzufuegen"

msgid "Edit"msgstr "Editieren"



Documentation

Importance of documentationExpand user base

User manual

Expand developer baseHelpful for hacking

Above all, documentation helps the original developerTendency to forget the purpose of old code

Writing documents perceived as bothersome

Documentation toolsAutomatically generate documents from source code

Can also graph class relationships, etc.Not designed to identify purpose of program



Doxygen, Javadoc, Doc++

Doxygen/Doc++Supports Java, C, C++, PHP, etc.Output in HTML or LaTeX formats

JavaDocComes standard with JDK

Sample output from Doxygen



Integrated Development Environments

Unix and GNU programming centers on command line tools

Convenience of many tools, but tools are difficult to master

Difficult to program outside of IDE framework

Demand for GUI-based IDEs as developer base growsSimplifies migration from Windows development environments

Leading IDEsEclipse with multi-language supportAnjuta for GNOME applicationsKDevelop for KDE applications



Eclipse

IDE written in JavaRuns on variety of operating systems

High extensibilityBilled as “universal tool platform”

Plug-in architecture to strengthen various featuresPlug-ins for Java, C, C++, PHP, Ruby and COBOL

History of EclipseIBM Visual Age released as FOSS

IBM sells Eclipse-based IBM Rational Software Development Platform



Bug Reporting Tools

Also called bug tracking systemsDedicated database for bug trackingWell-known tools: Bugzilla, Debian bug tracking system (BTS), GNU GNATS

Centralized management of bugsAs software grows in scale, management tools such as e-mail and spreadsheets become inadequate for grasping overall pictureStores information such as bug reporter, reproduction method, bug correction assignee, correction history, correction method, degree of importance, test status, etc.

Bug life cycle(1) bug report, (2) assignment of person to correct bug, (3) correction of bug, (4) testing and (5) close of bug reportBugs can sometimes recur after bug correction or during testing



Bugzilla

Developed by Mozilla.org to track bugs for Mozilla browser

Powerful bug tracking and search featuresUsed for projects such as XFree86, Apache, Samba, GNOME, etc.



(12) Maintaining Software and Development Examples

Updating software environmentsMaintaining software environments and configuration of softwareCompiling source code and installing buildsManagement of software packages

Software componentsEfficient use of software component enables us to reduce man-hour of developmentMajor software components

System building exampleExamples on Web System development and application software



Updating Software Environment

Maintaining software environmentEnvironment surrounding software changes day by day, due to new versions of library, software and protocols

Configuration of softwareSoftware needs to be set up to fit the surrounding environmentCompiling source code and installing builds

Binary packagesBinary package: software package that includes compiled software and the other files, in order to run correctly by putting into adequate directories

Package management toolsup2date, yum, apt-get



Maintaining Software Environments

Reasons for maintaining software environmentsSoftware updated on daily basis

Add or enhance featuresBug fixesPatch security holes

A lot of FOSS packages in particular are frequently updated

What happens when software is not maintainedProblem for yourself

Unstable operation or unauthorized access

Problem for others worldwideYour computer can be taken over and used to attack other computers or send spam mail



Configuration of Software

Elements that comprise softwareSource code alone is not enoughOther elements needed

Data used by softwareIcons and image dataOther data such as sample data, etc.

DocumentationManual pagesSupplementary information, update historyLicensing information

Configuration files

These files are typically circulated in one packagePackaging of source code



Compiling Source Code and Installing Builds

Common formats for source code distribution.tgz, .tar.gz or .tar.bz2 extensions

Tar utility for packing multiple files togetherTools for compressing and uncompressing files

Two sets of tools are frequently used, based on different compression formats

gzip / gunzip bzip2 / bunzip2

Procedure for unpacking and compiling source code and installing build

Unpack source code tar xzvf XXX.tgz

Configure source code to fit environment ./configure

Install build (Install as root) make; make install



Binary Packages

What is a binary package?Provided for each platform and contains pre-compiled and pre-configured binary code packaged with other necessary files

Examples of package management systemsRPM (Redhat Package Manager)

System developed by Red Hat

Deb: Package management system used by Debian

Difficult to apply binary packages to different environments

Often cannot be used with different versions of same Red Hat distribution, due to different library configurations, etc.If this happens, you can rebuild the binary package from the source package to install software



Package Management Tools

Main featuresUpdate software that has been revisedSimplify installation of new softwareAutomatically resolve dependencies (information about necessary libraries, etc.)

Leading package management toolsup2date

Developed for Red Hat Linux

yum (Yellow dog Updater, Modified)Developed for Yellow Dog Linux

apt-getDeveloped for Debian’s deb format

All three tools support RPM formatsGUI-based management tools also available such as Synaptic, etc.



Software Components

Software componentsEfficient to reduce man-hour of software development

Avoid “re-invention of the wheel”

Many components are published as FOSS

Finding and using componentsSeveral repositories are found in the Internet

Types of components (Data, XML, GUI, etc.)Classifies major components

Language specific componentsMajor language has its component library and the repository for them



Finding and Using Software Components

Finding software componentsStandard supplied components such as STL, PEAR, etc.Download from repository such as CPAN, RAA, etc.Find using search engine

Specialized libraries or those that have recently started development

Installing and using software componentsInclude in program source treeInstall to directories for each language

Often the case for scripting languagesSimplified installation using CPAN Perl module

Install as shared librariesInterface or binding to enable various languages to use library

After installation, library is called from program being created

# perl -MCPAN -e shellcpan> install [module-name]



Types of Components(Data Structure, XML, GUI, Etc.)

Data structureSTL

XMLLibxml, libxslt

GUIGTK+, Qt, Glade

Network

Web serviceOpenSOAP, Mono

GraphicsMesa, GLUT

Web pageHTMLParser

DatabaseODBC, JDBC

DeviceLibusb, v4l

MultimediaImlib, SDL

SecurityGnuPG (GPG), OpenSSL



Language Specific Components

Available libraries and specificationsJava

J2EE, StrutsJava Community Process (JCP)

Develops technology specifications

CMultitude of components such as GTK+, Qt, etc.XPG

APIs and commands to be provided by Unix

C++STL

Standard Template Library

PHPPEAR

http://pear.php.net/

Smarty http://smarty.php.net/

PerlCPAN

http://www.cpan.org/

PythonPython Cheese Shop (formerly PyPI)

http://www.python.org/pypi

RubyRAA

http://raa.ruby-lang.org/



System Building Example

Web SystemUsing software components such as PEAR, Mojavi, Smarty

ApplicationsApplication building exampleLocation of Application Files

1. Find files containing project information2. Examine subdirectories in source tree



Web System

PHP software componentsPEAR

Library for improving reusability of code

Mojavi MVC framework

SmartyTemplate engine

AdvantagesSimplifies support for different DBMSsSimple to reuse codeDivides labor of programmers and designers Linux

PHP

Smarty

Mojavi

Apache

DB

MySQL PostgreSQL

PEAR::DB

Controller

Model

View Template

HTML

Browser

etc...



Web System – PEAR::DB

PEAR::DBIntegrated interface for DBMSsPHP uses different functions for each DBMS

Changing DBMS requires major changes to code

Advantage of using PEAR::DB

Change just one part of code

Other classes for PEARAuthentication, testing, networking, XML, etc.

// Connection$connection = mysql_connect($host, $user, $pass);mysql_select_db($dbname);

// SQL execution$sql = "SELECT * FROM testtable";$result = mysql_query($sql);

// disconnectionmysql_close($connection);

// Declaration of PEAR::DBinclude("DB.php");

// Type of Database$type = "mysql";

// Connection$db = DB::connect

("$type://$user:$pass@$host/$dbname");

// SQL execution$sql = "SELECT * FROM testtable";$result = $db->query($sql);

// disconnection$db->disconnect();



Web System – Smarty

Template engineImproves design presentationIntuitive for designersDoes not affect input and output for Web design software

SmartyUsed for XOOPS and Rakuten (leading e-commerce site in Japan) systems

Many acceleration features

<h1>{$title}</h1><table>{section name=i loop=$array max=”10”} <tr><td>{$array[i]}</td></tr>{/section}</table>

<h1><? print $title; ?></h1>

<table><? for (i=0; $i<=10; i++) { print “<tr><td>”.$array[$i].”</td></tr>”;}?></table>



Application Building Example

Desktop application softwareExample: mpatrol

Tool for detecting memory leaks and illegal memory accessPlatforms supported

AIX, DG/UX, DRS/NX, DYNX/ptx, FreeBSD, HP/UX, IRIX, Red Hat Linux, SUSE Linux, LynxOS, SINIX, Solaris, Tru64, UnixWare, AmigaOS, Windows, etc.

Where to obtain mpatrolhttp://www.cbmamiga.demon.co.uk/mpatrol/

$ tar xzvf mpatrol_1.4.8.tar.gz...$ cd mpatrol$ lsAUTHORS COPYING.LIB NEWS VERSION config man testsCHECKSUMS ChangeLog README bin doc pkg toolsCOPYING INSTALL THANKS build extra src$



Location of Application Files

File configuration after unpackingAUTHORS Author’s name bin Location for binary code

CHECKSUMS Checksums for each file build

COPYING Copyright notice config

COPYING.LIB Copyright notices for libraries doc Contains documentation

ChangeLog Update history extra Contains files used for testing

INSTALL Installation information man Contains manuals

NEWS News pkg

README First file to read src Contains main source code

THANKS Acknowledgements tests Contains code for testing

VERSION Version information tools Contains source code for tools

Contains source code to generate necessary builds for each platform

Contains files used by Autotools

Contains files necessary for packaging such as for RPM, deb, etc.