file i/o: low-level

FILE I/O: Low-level

1

The Big Picture

2

Low-Level, cont. Some files are mixed format that are not readable by high-

level functions such as xlsread() [to be discussed]

Since the data is not easily recognized by the high-level function, every step to read the file requires a separate MATLAB command:1. Open a file, either to read-from or to write-to2. Read or write data from/to the file with the specific delimiters3. Close the file

3

Opening / Closing Files "Open a file"Program requests access to a file (from the OS) for reading

and/or writing data to/from the file.

"Close a file"Inform the OS that the program is finished working with the

file.

4

Opening a file: syntax Template to open a file

fh = fopen(<filename>,<mode>);

fh is known as a file handle or file identifier. It is used in future function calls to identify this is the file to use. It is like a "nickname" and we will use it instead of the filename when working with the file.

<filename> represents a string that is the name of the file with its extension (the letters after the dot). It can either be hardcoded, or within a variable

<mode> is a string specifying the purpose of opening the file – called the "access mode". The most commonly used are 'r' to read only from a file – bring data into memory from the disk 'w' to write to a file – put data from memory onto the disk 'a' to append to a file (add data to the end of the file) Add + to the string to combine reading and writing (e.g. 'r+', 'w+‘)

5

Closing a file: syntax Template to close a file

fclose(fh);

6

Some examples…% Example 1) open a file from which to readfileGrades = fopen('grades.txt', 'r'); %hardcode the filename<code block to be inserted here>%close filefclose(fileGrades);

% Example 2) ask user for a filename, then open it to readnameFile = input('Name of file with grades? (e.g. grades.txt): ', 's');fileGrades = fopen(nameFile, 'r'); <code block to be inserted here>%close filefclose(fileGrades);

7

Use the file handle – not the file name!

no quotes: a variable

Notice that the file handle variable can be any acceptable variable name

Closing Files: why? After working with a file, it is important to close the file.

Other than being good form, it is critical when writing to the file.

When the OS is supposed to put information on disk it frequently waits until it determines the best time. This is known as "write caching".

8

Closing Files: why? (.cont) You have seen write-caching with Window's "Safe

remove" warning on USB drives.

Windows may wait to write data. If your program finishes and Windows hasn't written this data, it will not be written at all!

Close the file before finishing the program - this forces Windows to write the data to the disk.

9

More Examples…

10

% Open a file for reading and writing

fh = fopen('my_project.abc', 'r+');

File name: The file extension (.abc here) is used by Windows – but only to tell it what program should be used if the Windows users wants to open the file.

You are free to use any extension you want with your data files. The only impact will be that Windows may not know what program should use that file.

Would 'w+' work also?

Maybe – see the next slide…

(1). Opening Files:Access-mode Codes The “access mode” codes indicate how you will be using

a file after you open it.

Since the operating system has permissions assigned to files, when you request access to a file you must tell the system in what mode you will be using the file.

The codes used tells the OS what it needs to know, and has an impact on how the file is used.

11

(1). Opening Files:File Position Pointer When a file is opened, a “file position pointer” is created.

The system keeps track of the point in the file to which the program has read or written.

Think of it like a cursor that moves as data is read or written to the file.

The file position pointer is set initially to different locations depending on the access mode!

12

(1). Opening Files:Access-mode Codes

Access Mode Initial Position

13

r, r+

w, w+

a, a+

Top (beginning) of file

Top (beginning) of file

Bottom (end) of file

(1). Opening Files:Access-mode & File Existence In addition to the file position pointer, the system also

has to decide what happens if the file does or does not exist initially?

If it already exists, should the file be deleted? If it doesn’t yet exist, should it be created?

14

(1). Opening Files:Access-mode & File ExistenceAccess Mode Delete? Create?

15

r, r+

w, w+

a, a+

No No

No

Yes Yes

Yes

You should be able to reason this out – memorization is not the key here!

QUIZ

A “log file” is a file that keeps a history of events. Many programs keep log files. They help programmers see what occurred in the past so that a problem can be fixed.

If your program is going to keep a log file, what is the best mode to use when opening this file? Why?

16

QUIZ You are writing a program that will manage a database.

You will be accessing files at different times within the program, so you decide to close and reopen the file several times. For each of these times, how should you open the file?

17

1. User wants to view a record in the database

2. User wants to modify a record in the database

3. User wants to add a record to the database

(2) Writing Text Filesfprintf(<file handle>, … The rest is as usual...);

Don’t forget the semi-colon!Otherwise, MATLAB displays in the command window a

number! fprintf() default output is how many characters were printed.

1818

File handle – not the file name!

Example:fh = fopen('log_file.txt', 'a');

fprintf(fh, 'Event #%d: \t%s\n', event_num, event_description);

MS Windows Text files When writing to a text file, MATLAB will write only a

single newline character to the end of a line – yet Windows requires two different characters there. So if you open the file in Notepad, it will not look like you expect:

19

MS Windows Text files There is nothing wrong with this – unless you intend to

work with the file outside of your program (and in Windows).

20

• To make it Windows-ready, write both a carriage return (\r) and a newline (\n):

(2) Writing Text Files Inserting data into the middle of a text file

Writing to text files is not like working in Word!

When you write to a text file, the data added to the file will write over any existing data in the file after the files position pointer – there is no “insert mode”!

21

(2) Writing Text Files

22

What we think should happen…

(2) Writing Text Files

23

What REALLY happens…

(2) Writing Text Files To avoid this problem…

You can’t.

You must write code that moves the existing file data so that you can insert the new data. This might mean copying to a new file, or looping and overwriting the old data.

24

(3) Reading text files

Reading an entire line as a string including storing the new line character in the variable

str = fgets(<file handle>); without storing the new line character in the variable

str = fgetl(<file handle>);

Reading numeric data data = fscanf(<file handle>);

25

Using fgets() Includes the new line character in your variable

26

Suppose we had this data file

And ran this program:

fh = fopen('testdata.txt', 'r');

x = fgets(fh);

fprintf('->%s<-', x);

Using fgets() Notice there are TWO newlines in the variable:

27

Using fgets() This is because Windows text files use two characters to

mark “end of line” (newline). Most other systems only use one character. MATLAB interprets both of these characters as newlines.

Fortunately, it’s easy to fix.

28

Using fgets()If you want to remove just one:x = x(1:end-1);

If you want to get rid of BOTH characters:x = x(1:end-2);

Or:x = strcat(x);

Or…29

Using fgetl() Reads past the newline, but DOES NOT include the

newline character in your variable

30

Using fscanf()fscanf() is like the reverse of fprintf() . You specify

the format you want to match and fscanf() will read from the file as long as it can match that format.

fscanf() is not good for reading strings because it will save the characters as their ASCII equivalents.

31

Using fscanf() Suppose we had this data file:

After opening the file, you could read the contents using:data = fscanf(fh, '%d\t%d')

32

Using fscanf() However, the result would be:

This demonstrates that fscanf() reads the data in line-order, but then stores it as a column. You can change this format using one more argument on the function call.

33

Using fscanf()Change the function call to:data = fscanf(fh, '%d\t%d', [2, 3])

And you get out:

34

Add this argument

MATLAB is still reading the data in line-order, and still storing the data in column-order. But we've now specified how big the columns will be – two rows each.

Using fscanf()But we may want the data to be in the form of the file. Unfortunately,

changing the third argument doesn’t help:

data = fscanf(fh, '%d\t%d', [3, 2])

Original file data:

This is because fscanf() is still filling the variable in “column-order” – it fills a column first and then moves onto the next column.

35

Using fscanf()To fix this, first read it in as a 2x3 matrix:

data = fscanf(fh, '%d\t%d', [2, 3])

Then transpose the matrix:data = data'

36

Using fscanf() But suppose we don’t know how many sets of data will

be in the file?

Use MATLAB’s inf constant. It means “as many as needed”

data = fscanf(fh, '%d\t%d', [2, inf])

Now, if the data file gets larger, your program can still handle it.

37Will this work? data = fscanf(fh, '%d\t%d', [inf, 2])

Wrapping Up Open a file: read? Write? Append?

fopen()

Close a file: fclose()

Read data: fgets(), fgetl(), fscanf()

38