CIS265/506 Cleveland State University – Prof. Victor Matos Chapter 17 Binary I/O CIS265/506 Cleveland State University – Prof. Victor Matos Adapted from: Introduction to Java Programming: Comprehensive Version, Eighth Edition by Y. Daniel Liang

Objectives To discover how I/O is processed in Java. To distinguish between text I/O and binary I/O. To read and write bytes using FileInputStream and FileOutputStream . To read and write primitive values and strings using DataInputStream/DataOutputStream. To store and restore objects using ObjectOutputStream and ObjectInputStream, and to understand how objects are serialized and what kind of objects can be serialized. To implement the Serializable interface to make objects serializable. To serialize arrays. To read and write the same file using the RandomAccessFile class.

Motivations Data stored in a text file is represented in human-readable form. Data stored in a binary file is represented in binary form. Binary files are designed to be read by programs. For example, Java source programs are stored in text files and can be read by a text editor, but Java classes are stored in binary files and are read by the JVM. The advantage of binary files is that they are more efficient to process than text files.

How is I/O Handled in Java? A File object encapsulates the properties of a file or a path, but does not contain the methods for reading/writing data from/to a file. To perform IO operations you need to create objects using appropriate Java I/O classes. Scanner input = new Scanner( new File("c:\\temp.txt") ); System.out.println( input.nextLine() ); PrintWriter output = new PrintWriter("c:\\temp.txt"); output.println("Java 101"); output.close();

Text File vs. Binary File Observation 1 Although it is not entirely technically correct, you can imagine that a text file consists of a sequence of characters and a binary file consists of a sequence of bits. For example, the decimal integer 199 is stored as the sequence of three characters: '1', '9', '9' in a text file. The same integer is stored as a byte-type value C7 in a binary file, because decimal 199 equals to hex C7 (Binary: 11000111).

Binary I/O Text I/O requires encoding and decoding. The JVM converts a Unicode symbols to/from a file specific encoding when reading and writing. Binary I/O does not require conversions. When you write a byte to a file, the original byte is copied into the file. When you read a byte from a file, the exact byte in the file is returned.

Binary I/O Classes A stream is an abstraction that either produces or consumes data. Streams are sequential in nature. In Java streams move two possible data types: bytes and characters 

InputStream

OutputStream

FileInputStream/FileOutputStream Associates a binary input/output stream with an external file. All the methods in FileInputStream/FileOuptputStream are inherited from their superclasses.

Example1: FileInputStream FileInputStream is for reading/writing bytes from/to a disk file. public static void main(String[] args) throws IOException { FileInputStream fis1 = new FileInputStream("c:\\temp\\mydata.txt"); //alternatively you may also say... File file = new File("c:\\temp\\mydata.txt"); FileInputStream fis2 = new FileInputStream( file ); int avail = fis1.available(); for(int i=0; i<avail; i++ ){ int data = fis1.read(); System.out.print(data ); } }//main 13 Carriage Return 10 Line Feed Disk available 27 CONSOLE 65 13 10 66 13 10 67 13 10 97 13 10 98 13 10 99 13 10 48 13 10 49 13 10 50 13 10 A B C a b c 0 1 2

Example2: FileInputStream & FileOutputStream import java.io.*; public class TestFileStream { public static void main(String[] args) throws IOException { // Create an output stream to the file FileOutputStream output = new FileOutputStream("temp.dat"); output.write('A'); output.write('B'); output.write('a'); output.write('b'); output.write(255); output.write(256); // Output values to the file for (int i = 0; i <= 20; i++) output.write(i); // Close the output stream output.close(); // Create an input stream for the file FileInputStream input = new FileInputStream("temp.dat"); // Read values from the file int value; while ((value = input.read()) != -1) System.out.print(value + " "); input.close(); } Disk File CONSOLE 65 66 97 98 255 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 A, B, a, b, 255, 256 0, 1, 2, … , 20

Example2: FileInputStream & FileOutputStream Continuation… Viewing temp.dat file with MS-Lister Viewing temp.dat file with hexadecimal editor HxD Hexedit. CONSOLE 65 66 97 98 255 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 A, B, a, b, 255, 256 0, 1, 2, … , 20 Hex 41 = Binary 1000 0001 = int 65 = Char ‘A’ 4 1

FileOutputStream To construct a FileOutputStream, use the following constructors: public FileOutputStream ( String filename ) public FileOutputStream ( File file ) public FileOutputStream ( String filename, boolean append ) public FileOutputStream ( File file, boolean append )    If the file does not exist, a new file would be created. If the file already exists, the first two constructors would delete the current contents in the file. To retain the current content and append new data into the file, use the last two constructors by passing true to the append parameter.

FilterInputStream/FilterOutputStream Using a filter class enables you to read integers, doubles, and strings instead of just bytes and characters. FilterInputStream and FilterOutputStream are the base classes for filtering data. When you need to process primitive numeric types, use DatInputStream and DataOutputStream to filter bytes.

DataInputStream/DataOutputStream DataInputStream reads bytes from the stream and converts them into appropriate primitive type values or strings. DataOutputStream converts primitive type values or strings into bytes and output the bytes to the stream.

DataInputStream DataInputStream reads bytes from the stream and converts them into appropriate primitive type values or strings.

DataOutputStream DataOutputStream converts primitive type values or strings into bytes and outputs the bytes to the stream.

Characters and Strings in Binary I/O Unicode is an industry standard for dealing with written text. It includes over 1,1Million Code Points (u+XXXXXX) and 100+ scripts such a Latin, Arabic, Hebrew, Greek, Cyrillic, Chinese, Thai, Indic, Cherokee, Braille, … Unicode can be implemented by different character encodings (such as UTF-8) writeChars(String s) writes the Unicode value for each character in the string s to the output (2-bytes for each symbol). writeUTF(String s) writes the Unicode value of each character in the string s to the output. Interesting Links: http://farmdev.com/talks/unicode/ http://www.unicode.org/

Characters and Strings in Binary I/O Plain Text Unicode Hello U+0048 U+0065 U+006C U+006C U+006F U+8272 U+8377 Traditional Chinese String str = "\u8272\u8377"; char c = '\u8377'; Character letter = new Character( '\u0048');

Characters and Strings in Binary I/O You may use the MS-Windows charmap tool to manage Unicodes. Scripts Code Points

Characters and Strings in Binary I/O What is UTF-8 ? Java, Microsoft.NET, XML and most modern Operating Systems and databases use Unicode (instead of ANSI). UTF-8 is an alternative encoding scheme that stores a character using 1, 2, or 3 bytes. ASCII values (less than 0x7F) are coded in one byte. Unicode values less than 0x7FF are coded in two bytes. Other Unicode values are coded in three bytes. 0x7F 0x7FF > 0x7FF 127 2047 > 2047 See http://www.utf8-chartable.de/

Using DataInputStream/DataOutputStream The statements given below create data streams. The first statement creates an input stream for file in.dat; the second statement creates an output stream for file out.dat. DataInputStream infile = new DataInputStream( new FileInputStream("in.dat")); DataOutputStream outfile = new DataOutputStream ( new FileOutputStream("out.dat"));

Example3: Using DataInputStream/DataOutputStream Write then read a Chinese symbol in Unicode. Plain Text (Chinese) Unicode – Code Point (Hexadecimal) (Decimal) U + 8272 33394 // Create an output stream for file temp.dat DataOutputStream output = new DataOutputStream( new FileOutputStream("temp2.dat") ); String str = "\u8272"; // character in traditional Chinese output.writeUTF(str); output.close(); // Create an input stream for file temp.dat DataInputStream input = new DataInputStream( new FileInputStream("temp2.dat") ); String str2 = input.readUTF(); int codePoint = str2.codePointAt(0); System.out.printf(" %s %d %x ", str2, codePoint, codePoint); Console 色 33394 8272

Example4: Using DataInputStream/DataOutputStream Write then read student’s name & score. public class TestDataStream { public static void main(String[] args) throws IOException { // Create an output stream for file temp.dat DataOutputStream output = new DataOutputStream( new FileOutputStream("temp2.dat") ); output.writeUTF("John"); output.writeDouble(85.5); output.writeUTF("Jim"); output.writeDouble(185.5); output.writeUTF("George"); output.writeDouble(105.25); // Close output stream output.close(); // Create an input stream for file temp.dat DataInputStream input = new DataInputStream( new FileInputStream("temp2.dat") ); // Read student test scores from the file System.out.println(input.readUTF() + " " + input.readDouble()); } Console John 85.5 Jim 185.5 George 105.25

Read/Write Ops. - Order and Format CAUTION: You have to read the data in the same order and same format in which they are stored. For example, when names are written in UTF-8 using writeUTF, you must read names using readUTF. Checking End of File TIP: If you keep reading data at the end of a stream, an EOFException would occur. You can use input.available() to check for End-Of-File condition. Expression input.available() == 0 indicates that it is the end of a file.

Eclipse Console & Unicode Symbols How to show Unicode symbols on your Eclipse Console? References: http://bytecounts.com/blog/?p=28 http://paranoid-engineering.blogspot.com/2008/05/getting-unicode-output-in-eclipse.html Three steps operation: Install an appropriate Unicode Font/Script for your Console and Change your JRE configuration. Make sure the current Eclipse “Run Configuration” is set for UTF-8 Details Go to Eclipse Menu: Window > Preferences > Java > Installed JREs > select JRE > select Edit. Add to [Default VM Arguments]: -Dfile.encoding=UTF-8 Go to Eclipse Menu: Window > Preferences > General > Appearance > Colors & Fonts > Debug > Console font > Edit. Choose a Unicode font, like Lucinda Sans Unicode Note: You may use Microsoft-JhengHei for Chinese-Japanese-Korean (CJK) ideographs. By default fonts not in your language-base are disable. You may have to install/enable them.

Eclipse Console & Unicode Symbols How to show Unicode symbols on your Eclipse Console? Step 3: Fixing your: “Run Configuration” to use UTF-8 encoding instead of Cp1252 default encoding

Eclipse Console & Unicode Symbols How to show Unicode symbols on your Eclipse Console? After making changes suggested in previous page, we could add the following fragment to Example4 and obtain results listed below. // Greek symbols: Koppa, Omega, alpha System.out.println("\u03d8\u03a0\u03b1"); // CJK (Chinese, Japanese, Korean) System.out.println("\u8272\u8377\u322F\u5193"); //exclamation, space, numeral System.out.println("\u0021\u0020\u0023"); // numbers, lower, upper case Latin System.out.println("\u0035\u0036\u0061\u0062\u0041\u0042"); CONSOLE ϘΩα 色荷㈯冓 ! # 56abAB

Example5: Eclipse Console & Unicode Symbols ϘΩα Ϙ 0984 Ω 0937 α 0945 色荷㈯冓 色 33394 荷 33655 ㈯ 12847 冓 20883 ! # ! 0033 0032 # 0035 56abAB 5 0053 6 0054 a 0097 b 0098 A 0065 B 0066 Señorita S 0083 e 0101 ñ 0241 o 0111 r 0114 i 0105 t 0116 Extending Example4. Using writeUTF() / readUTF() methods public class TestUnicode { public static void main(String[] args) throws IOException { // Create an output stream for file temp3.dat String strGreek = "\u03d8\u03a9\u03b1"; String strCJK = "\u8272\u8377\u322F\u5193"; String strPunctuation = "\u0021\u0020\u0023"; String strLatin = "\u0035\u0036\u0061\u0062\u0041\u0042"; String strLatin2 = "Se\u00F1orita"; DataOutputStream output = new DataOutputStream(new FileOutputStream ( "temp3.dat“ ) ); output.writeUTF( strGreek ); output.writeUTF( strCJK ); output.writeUTF( strPunctuation ); output.writeUTF( strLatin ); output.writeUTF( strLatin2 ); // Close output stream output.close(); // Create an input stream for file temp3.dat DataInputStream input = new DataInputStream(new FileInputStream( "temp3.dat“ ) ); // Read UTF strings from file, show strings & CodePoints as char and bytes while (input.available() > 0 ){ String str = input.readUTF(); System.out.printf( "\n %s" , str ); for(int i=0; i<str.length(); i++) { int codePoint = str.codePointAt(i); System.out.printf( "\n %c %04d " , codePoint, codePoint ); }

BufferedInputStream / BufferedOutputStream WHY ? BufferedInputStream/ BufferedOutputStream can be used to speed up input and output by reducing the number of reads and writes. When the BufferedInputStream is created, an internal buffer array is created. As bytes from the stream are read or skipped, the internal buffer is refilled as necessary from the contained input stream, many bytes at a time. The mark operation remembers a point in the input stream and the reset operation causes all the bytes read since the most recent mark operation to be reread before new bytes are taken from the contained input stream. See http://docs.oracle.com/javase/1.4.2/docs/api/java/io/BufferedInputStream.html

BufferedInputStream / BufferedOutputStream Using buffers to speed up I/O All the methods BufferedInputStream/ BufferedOutputStream are inherited from the InputStream/OutputStream classes.

Constructing BufferedInputStream/BufferedOutputStream // Create a BufferedInputStream public BufferedInputStream ( InputStream in) public BufferedInputStream ( InputStream in, int bufferSize ) // Create a BufferedOutputStream public BufferedOutputStream ( OutputStream out) public BufferedOutputStream ( OutputStreamr out, int bufferSize ) If no buffer size is specified, the default size is 512 bytes. A buffered input stream reads as many data as possible into its buffer in a single read call. By contrast, a buffered output stream calls the write method only when its buffer fills up or when the flush() method is called.

Case Study: Copy File Project The program copies a source file to a destination & displays the number of bytes read. If the source does not exist, it tells the user the file is not found. If the target file already exists, tell the user the file already exists. public class Copy { public static void main(String[] args) throws IOException { // Check command-line parameter usage // String [] argsTest = { "c:\\Windows\\explorer.exe", "c:\\temp\\explorer-COPY.exe" }; // args = argsTest; // use these values for testing if (args.length != 2) { System.out.println( "Usage: java Copy sourceFile targetfile"); System.exit(0); } // Check if source file exists File sourceFile = new File(args[0]); if (!sourceFile.exists()) { System.out.println("Source file " + args[0] + " not exist"); // Check if target file exists File targetFile = new File(args[1]); if (targetFile.exists()) { System.out.println("Target file " + args[1] + " already exists");

Case Study: Copy File Project 2 of 2 . Continuation… // Create an input stream BufferedInputStream input = new BufferedInputStream(new FileInputStream(sourceFile)); // Create an output stream BufferedOutputStream output = new BufferedOutputStream(new FileOutputStream(targetFile)); // Continuously read a byte from input and write it to output int r; int numberOfBytesCopied = 0; while ((r = input.read()) != -1) { output.write((byte)r); numberOfBytesCopied++; } // Close streams input.close(); output.close(); // Display the file size System.out.println(numberOfBytesCopied + " bytes copied");

Object I/O ObjectInputStream/ObjectOutputStream enables you to perform I/O for objects in addition to primitive type values and strings. You can replace DataInputStream/DataOutputStream completely with ObjectInputStream/ObjectOutputStream.

ObjectInputStream This class is typically used to serialize objects residing in volatile memory and transfer them to persistent storage (such as disk) The method writeObject(…) is used to convert the object’s state into a sequence of bytes.

ObjectInputStream This class is typically used to de-serialize persistent objects residing in disk and bring them back to memory. The readObject(…) method is used to convert a sequence of bytes into an object’s instance.

Using Object Streams (Output) public class TestObjectOutputStream { public static void main(String[] args) throws IOException { // Create an output stream for file object.dat ObjectOutputStream output = new ObjectOutputStream( new FileOutputStream ( "object.dat“ ) ); // Write a string, double value, and object to the file output.writeUTF ( "John" ); output.writeDouble ( 85.5 ); output.writeObject ( new java.util.Date() ); // Close output stream output.close(); }

Using Object Streams (Input) public class TestObjectInputStream { public static void main(String[] args) throws ClassNotFoundException, IOException { // Create an input stream for file object.dat ObjectInputStream input = new ObjectInputStream(new FileInputStream("object.dat")); // Write a string, double value, and object to the file String name = input.readUTF(); double score = input.readDouble(); java.util.Date date = (java.util.Date)(input.readObject()); System.out.println(name + " " + score + " " + date); // Close output stream input.close(); } CONSOLE John 85.5 Fri Feb 24 20:51:25 EST 2012

The Serializable Interface Objects that can be written to an object stream are said to be serializable. Some classes such as Thread, Socket, and Streams are not serializable. The class of a serializable object must implement (or inherit) Serializable. The Serializable interface is a marker interface. It has no methods. Fortunately most Java API classes, utilities, lists, and GUI components implement the Serializable interface Example Person p1 = new Person(“Maria Macarena”, 28, “123 Salsa Rd. Cleveland OH”); . . . outputFile.writeObject ( p1 ); // this is simpler than writing to disk // each data item inside the p1 object.

The transient Keyword Question: If an object is an instance of Serializable, but it contains non-serializable fields, can the object be serialized? The answer is no ! To enable the object to be serialized, you can use the transient keyword to mark these data fields to tell the JVM to ignore these fields when writing the object to an object stream.

The transient Keyword, cont. Consider the following class:   public class Foo implements java.io.Serializable { private int v1; private static double v2; private transient A v3 = new A(); } class A { ... } // assume A is not serializable When an object of the Foo class is serialized, only variable v1 is serialized. Variable v2 is not serialized because it is a static variable, and variable v3 is not serialized because it is marked transient. If v3 were not marked transient, a java.io.NotSerializableException would occur.

Example6: Serializing Arrays 1 of 2 An array is serializable if all its elements are serializable. So an entire array can be saved using a single writeObject into a file and later restored using readObject. public class TestSerializableArray implements Serializable { public static void main(String[] args) throws FileNotFoundException, IOException, ClassNotFoundException { // create list, populate & save it to disk ArrayList<Person> list = new ArrayList<Person>(); list.add(new Person("Miles VorKossigan", 28)); list.add(new Person("Honor Harrington", 29)); list.add(new Person("Albin Maker", 20)); list.add(new Person("Ender Higgins", 10)); ObjectOutputStream outputFile = new ObjectOutputStream( new FileOutputStream( "temp5.dat“ ) ); outputFile.writeObject(list); outputFile.close(); list.clear(); // read list back from disk ObjectInputStream inputFile = new ObjectInputStream( new FileInputStream( "temp5.dat“ ) ); list = (ArrayList<Person>) inputFile.readObject(); for (Person p : list){ System.out.println( p.toString() ); } inputFile.close();

Example6: Serializing Arrays 2 of 2 public class Person implements Serializable { // class variables private String mPersonName; private int mPersonAge; // constructor(s) public Person(String mPersonName, int mPersonAge) { this.mPersonName = mPersonName; this.mPersonAge = mPersonAge; } // mutators public String getmPersonName() { return mPersonName; public void setmPersonName(String mPersonName) { public int getmPersonAge() { return mPersonAge; public void setmPersonAge(int mPersonAge) { // custom methods public String toString(){ String result = " Name: " + this.getmPersonName() + " Age: " + this.getmPersonAge(); return result;

The RandomAccessFile Class All of the streams we have used so far are known as read- only or write- only streams. The external container of these streams are sequential files that cannot be updated without creating a new file. It is often necessary to change data in the files by inserting, deleting, or re-writing records. Java provides the RandomAccessFile class to allow a file to be read from and written to at random locations.

File Pointer A random access file consists of a sequence of bytes. There is a special marker called file pointer that is positioned at one of these bytes. A read or write operation takes place at the location of the file pointer. When a file is opened, the file pointer sets at the beginning of the file. When you read or write data to the file, the file pointer moves forward to the next data.

RandomAccessFile

RandomAccessFile Methods void seek(long pos) throws IOException; Sets the offset from the beginning of the RandomAccessFile stream to where the next read or write occurs. long getFilePointer() throws IOException; Returns the current offset, in bytes, from the beginning of the file to where the next read or write occurs.

RandomAccessFile Methods, cont. long length()IOException Returns the length of the file. final void writeChar(int v) throws IOException Writes a character to the file as a two-byte Unicode, with the high byte written first. final void writeChars(String s)throws IOException Writes a string to the file as a sequence of characters.

RandomAccessFile Constructor // allows read and write RandomAccessFile raf = new RandomAccessFile( "test.dat", "rw" ); // read only RandomAccessFile raf = new RandomAccessFile( "test.dat", "r" );

Example7: A Simple RandomAccessFile public class TestRandomAccessFile { public static void main(String[] args) throws IOException { // Create a random access file RandomAccessFile inout = new RandomAccessFile("inout.dat", "rw"); // Clear the file to destroy the old contents if exists inout.setLength(0); // Write new integers to the file for (int i = 0; i < 200; i++) inout.writeInt(i); // Display the current length of the file System.out.println("Current file length is " + inout.length()); // Retrieve the first number inout.seek(0); // Move the file pointer to the beginning System.out.println("The first number is " + inout.readInt()); // Retrieve the second number inout.seek(1 * 4); // Move the file pointer to the second number System.out.println("The second number is " + inout.readInt()); // Retrieve the tenth number inout.seek(9 * 4); // Move the file pointer to the tenth number System.out.println("The tenth number is " + inout.readInt()); Console Current file length is 800 The first number is 0 The second number is 1 The tenth number is 9

Example7: A Simple RandomAccessFile // Modify the eleventh number inout.writeInt(555); // Append a new number inout.seek(inout.length()); // Move the file pointer to the end inout.writeInt(999); // Display the new length System.out.println("The new length is " + inout.length()); // Retrieve the new eleventh number inout.seek(10 * 4); // Move the file pointer to the eleventh number System.out.println("The eleventh number is " + inout.readInt()); inout.close(); } Console Current file length is 800 The first number is 0 The second number is 1 The tenth number is 9 The new length is 804 The eleventh number is 555

Appendix 1: Fixed Length String I/O Random access files are often used to process files of records. For convenience, fixed-length records are used in random access files so that a record can be located easily. A record consists of a fixed number of fields. A field can be a string or a primitive data type. A string in a fixed-length record has a maximum size. If a string is smaller than the maximum size, the rest of the string is padded with blanks.