devguide/coding/file-io.html

{{+bindTo:partials.standard_nacl_article}}

<section id="file-i-o">
<span id="devguide-coding-fileio"></span><h1 id="file-i-o"><span id="devguide-coding-fileio"></span>File I/O</h1>
<div class="contents local" id="contents" style="display: none">
<ul class="small-gap">
<li><a class="reference internal" href="#introduction" id="id2">Introduction</a></li>
<li><a class="reference internal" href="#reference-information" id="id3">Reference information</a></li>
<li><p class="first"><a class="reference internal" href="#local-file-i-o" id="id4">Local file I/O</a></p>
<ul class="small-gap">
<li><a class="reference internal" href="#enabling-local-file-i-o" id="id5">Enabling local file I/O</a></li>
<li><a class="reference internal" href="#testing-local-file-i-o" id="id6">Testing local file I/O</a></li>
</ul>
</li>
<li><p class="first"><a class="reference internal" href="#the-file-io-example" id="id7">The <code>file_io</code> example</a></p>
<ul class="small-gap">
<li><a class="reference internal" href="#file-i-o-overview" id="id8">File I/O overview</a></li>
<li><a class="reference internal" href="#creating-and-writing-a-file" id="id9">Creating and writing a file</a></li>
<li><a class="reference internal" href="#opening-and-reading-a-file" id="id10">Opening and reading a file</a></li>
<li><a class="reference internal" href="#deleting-a-file" id="id11">Deleting a file</a></li>
<li><a class="reference internal" href="#making-a-directory" id="id12">Making a directory</a></li>
<li><a class="reference internal" href="#listing-the-contents-of-a-directory" id="id13">Listing the contents of a directory</a></li>
</ul>
</li>
<li><p class="first"><a class="reference internal" href="#file-io-deep-dive" id="id14"><code>file_io</code> deep dive</a></p>
<ul class="small-gap">
<li><a class="reference internal" href="#opening-a-file-system-and-preparing-for-file-i-o" id="id15">Opening a file system and preparing for file I/O</a></li>
<li><a class="reference internal" href="#handling-messages-from-javascript" id="id16">Handling messages from JavaScript</a></li>
<li><a class="reference internal" href="#saving-a-file" id="id17">Saving a file</a></li>
<li><a class="reference internal" href="#loading-a-file" id="id18">Loading a file</a></li>
<li><a class="reference internal" href="#id1" id="id19">Deleting a file</a></li>
<li><a class="reference internal" href="#listing-files-in-a-directory" id="id20">Listing files in a directory</a></li>
<li><a class="reference internal" href="#making-a-new-directory" id="id21">Making a new directory</a></li>
</ul>
</li>
</ul>

</div><section id="introduction">
<h2 id="introduction">Introduction</h2>
<p>This chapter describes how to use the <a class="reference external" href="/native-client/pepper_stable/cpp/classpp_1_1_file_i_o">FileIO API</a> to read and write
files using a local secure data store.</p>
<p>You might use the File IO API with the URL Loading APIs to create an overall
data download and caching solution for your NaCl applications. For example:</p>
<ol class="arabic simple">
<li>Use the File IO APIs to check the local disk to see if a file exists that
your program needs.</li>
<li>If the file exists locally, load it into memory using the File IO API. If
the file doesn&#8217;t exist locally, use the URL Loading API to retrieve the
file from the server.</li>
<li>Use the File IO API to write the file to disk.</li>
<li>Load the file into memory using the File IO API when needed by your
application.</li>
</ol>
<p>The example discussed in this chapter is included in the SDK in the directory
<code>examples/api/file_io</code>.</p>
</section><section id="reference-information">
<h2 id="reference-information">Reference information</h2>
<p>For reference information related to FileIO, see the following documentation:</p>
<ul class="small-gap">
<li><a class="reference external" href="/native-client/pepper_stable/cpp/file__io_8h">file_io.h</a> - API to create a
FileIO object</li>
<li><a class="reference external" href="/native-client/pepper_stable/cpp/file__ref_8h">file_ref.h</a> - API to create
a file reference or &#8220;weak pointer&#8221; to a file in a file system</li>
<li><a class="reference external" href="/native-client/pepper_stable/cpp/file__system_8h">file_system.h</a> - API to
create a file system associated with a file</li>
</ul>
</section><section id="local-file-i-o">
<h2 id="local-file-i-o">Local file I/O</h2>
<p>Chrome provides an obfuscated, restricted area on disk to which a web app can
safely <a class="reference external" href="https://developers.google.com/chrome/whitepapers/storage#persistent">read and write files</a>. The
Pepper FileIO, FileRef, and FileSystem APIs (collectively called the File IO
APIs) allow you to access this sandboxed local disk so you can read and write
files and manage caching yourself. The data is persistent between launches of
Chrome, and is not removed unless your application deletes it or the user
manually deletes it. There is no limit to the amount of local data you can
use, other than the actual space available on the local drive.</p>
<section id="enabling-local-file-i-o">
<span id="enabling-file-access"></span><span id="quota-management"></span><h3 id="enabling-local-file-i-o"><span id="enabling-file-access"></span><span id="quota-management"></span>Enabling local file I/O</h3>
<p>The easiest way to enable the writing of persistent local data is to include
the <a class="reference external" href="/extensions/declare_permissions#unlimitedStorage">unlimitedStorage permission</a> in your Chrome Web Store
manifest file. With this permission you can use the Pepper FileIO API without
the need to request disk space at run time. When the user installs the app
Chrome displays a message announcing that the app writes to the local disk.</p>
<p>If you do not use the <code>unlimitedStorage</code> permission you must include
JavaScript code that calls the <a class="reference external" href="http://updates.html5rocks.com/2011/11/Quota-Management-API-Fast-Facts">HTML5 Quota Management API</a> to
explicitly request local disk space before using the FileIO API. In this case
Chrome will prompt the user to accept a requestQuota call every time one is
made.</p>
</section><section id="testing-local-file-i-o">
<h3 id="testing-local-file-i-o">Testing local file I/O</h3>
<p>You should be aware that using the <code>unlimitedStorage</code> manifest permission
constrains the way you can test your app. Three of the four techniques
described in <a class="reference internal" href="/native-client/devguide/devcycle/running.html"><em>Running Native Client Applications</em></a>
read the Chrome Web Store manifest file and enable the <code>unlimitedStorage</code>
permission when it appears, but the first technique (local server) does not.
If you want to test the file IO portion of your app with a simple local server,
you need to include JavaScript code that calls the HTML5 Quota Management API.
When you deliver your application you can replace this code with the
<code>unlimitedStorage</code> manifest permission.</p>
</section></section><section id="the-file-io-example">
<h2 id="the-file-io-example">The <code>file_io</code> example</h2>
<p>The Native Client SDK includes an example, <code>file_io</code>, that demonstrates how
to read and write a local disk file. Since you will probably run the example
from a local server without a Chrome Web Store manifest file, the example&#8217;s
index file uses JavaScript to perform the Quota Management setup as described
above. The example has these primary files:</p>
<ul class="small-gap">
<li><code>index.html</code> - The HTML code that launches the Native Client module and
displays the user interface.</li>
<li><code>example.js</code> - JavaScript code that requests quota (as described above). It
also listens for user interaction with the user interface, and forwards the
requests to the Native Client module.</li>
<li><code>file_io.cc</code> - The code that sets up and provides an entry point to the
Native Client module.</li>
</ul>
<p>The remainder of this section covers the code in the <code>file_io.cc</code> file for
reading and writing files.</p>
<section id="file-i-o-overview">
<h3 id="file-i-o-overview">File I/O overview</h3>
<p>Like many Pepper APIs, the File IO API includes a set of methods that execute
asynchronously and that invoke callback functions in your Native Client module.
Unlike most other examples, the <code>file_io</code> example also demonstrates how to
make Pepper calls synchronously on a worker thread.</p>
<p>It is illegal to make blocking calls to Pepper on the module&#8217;s main thread.
This restriction is lifted when running on a worker thread&#8212;this is called
&#8220;calling Pepper off the main thread&#8221;. This often simplifies the logic of your
code; multiple asynchronous Pepper functions can be called from one function on
your worker thread, so you can use the stack and standard control flow
structures normally.</p>
<p>The high-level flow for the <code>file_io</code> example is described below.  Note that
methods in the namespace <code>pp</code> are part of the Pepper C++ API.</p>
</section><section id="creating-and-writing-a-file">
<h3 id="creating-and-writing-a-file">Creating and writing a file</h3>
<p>Following are the high-level steps involved in creating and writing to a
file:</p>
<ol class="arabic simple">
<li><code>pp::FileIO::Open</code> is called with the <code>PP_FILEOPEN_FLAG_CREATE</code> flag to
create a file.  Because the callback function is <code>pp::BlockUntilComplete</code>,
this thread is blocked until <code>Open</code> succeeds or fails.</li>
<li><code>pp::FileIO::Write</code> is called to write the contents. Again, the thread is
blocked until the call to <code>Write</code> completes. If there is more data to
write, <code>Write</code> is called again.</li>
<li>When there is no more data to write, call <code>pp::FileIO::Flush</code>.</li>
</ol>
</section><section id="opening-and-reading-a-file">
<h3 id="opening-and-reading-a-file">Opening and reading a file</h3>
<p>Following are the high-level steps involved in opening and reading a file:</p>
<ol class="arabic simple">
<li><code>pp::FileIO::Open</code> is called to open the file. Because the callback
function is <code>pp::BlockUntilComplete</code>, this thread is blocked until Open
succeeds or fails.</li>
<li><code>pp::FileIO::Query</code> is called to query information about the file, such as
its file size. The thread is blocked until <code>Query</code> completes.</li>
<li><code>pp::FileIO::Read</code> is called to read the contents. The thread is blocked
until <code>Read</code> completes. If there is more data to read, <code>Read</code> is called
again.</li>
</ol>
</section><section id="deleting-a-file">
<h3 id="deleting-a-file">Deleting a file</h3>
<p>Deleting a file is straightforward: call <code>pp::FileRef::Delete</code>. The thread is
blocked until <code>Delete</code> completes.</p>
</section><section id="making-a-directory">
<h3 id="making-a-directory">Making a directory</h3>
<p>Making a directory is also straightforward: call <code>pp::File::MakeDirectory</code>.
The thread is blocked until <code>MakeDirectory</code> completes.</p>
</section><section id="listing-the-contents-of-a-directory">
<h3 id="listing-the-contents-of-a-directory">Listing the contents of a directory</h3>
<p>Following are the high-level steps involved in listing a directory:</p>
<ol class="arabic simple">
<li><code>pp::FileRef::ReadDirectoryEntries</code> is called, and given a directory entry
to list. A callback is given as well; many of the other functions use
<code>pp::BlockUntilComplete</code>, but <code>ReadDirectoryEntries</code> returns results in
its callback, so it must be specified.</li>
<li>When the call to <code>ReadDirectoryEntries</code> completes, it calls
<code>ListCallback</code> which packages up the results into a string message, and
sends it to JavaScript.</li>
</ol>
</section></section><section id="file-io-deep-dive">
<h2 id="file-io-deep-dive"><code>file_io</code> deep dive</h2>
<p>The <code>file_io</code> example displays a user interface with a couple of fields and
several buttons. Following is a screenshot of the <code>file_io</code> example:</p>
<img alt="/native-client/images/fileioexample.png" src="/native-client/images/fileioexample.png" />
<p>Each radio button is a file operation you can perform, with some reasonable
default values for filenames. Try typing a message in the large input box and
clicking <code>Save</code>, then switching to the <code>Load File</code> operation, and
clicking <code>Load</code>.</p>
<p>Let&#8217;s take a look at what is going on under the hood.</p>
<section id="opening-a-file-system-and-preparing-for-file-i-o">
<h3 id="opening-a-file-system-and-preparing-for-file-i-o">Opening a file system and preparing for file I/O</h3>
<p><code>pp::Instance::Init</code> is called when an instance of a module is created. In
this example, <code>Init</code> starts a new thread (via the <code>pp::SimpleThread</code>
class), and tells it to open the filesystem:</p>
<pre class="prettyprint">
virtual bool Init(uint32_t /*argc*/,
                  const char * /*argn*/ [],
                  const char * /*argv*/ []) {
  file_thread_.Start();
  // Open the file system on the file_thread_. Since this is the first
  // operation we perform there, and because we do everything on the
  // file_thread_ synchronously, this ensures that the FileSystem is open
  // before any FileIO operations execute.
  file_thread_.message_loop().PostWork(
      callback_factory_.NewCallback(&amp;FileIoInstance::OpenFileSystem));
  return true;
}
</pre>
<p>When the file thread starts running, it will call <code>OpenFileSystem</code>. This
calls <code>pp::FileSystem::Open</code> and blocks the file thread until the function
returns.</p>
<aside class="note">
Note that the call to <code>pp::FileSystem::Open</code> uses
<code>pp::BlockUntilComplete</code> as its callback. This is only possible because we
are running off the main thread; if you try to make a blocking call from the
main thread, the function will return the error
<code>PP_ERROR_BLOCKS_MAIN_THREAD</code>.
</aside>
<pre class="prettyprint">
void OpenFileSystem(int32_t /*result*/) {
  int32_t rv = file_system_.Open(1024 * 1024, pp::BlockUntilComplete());
  if (rv == PP_OK) {
    file_system_ready_ = true;
    // Notify the user interface that we're ready
    PostMessage(&quot;READY|&quot;);
  } else {
    ShowErrorMessage(&quot;Failed to open file system&quot;, rv);
  }
}
</pre>
</section><section id="handling-messages-from-javascript">
<h3 id="handling-messages-from-javascript">Handling messages from JavaScript</h3>
<p>When you click the <code>Save</code> button, JavaScript posts a message to the NaCl
module with the file operation to perform sent as a string (See <a class="reference internal" href="/native-client/devguide/coding/message-system.html"><em>Messaging
System</em></a> for more details on message passing). The string is
parsed by <code>HandleMessage</code>, and new work is added to the file thread:</p>
<pre class="prettyprint">
virtual void HandleMessage(const pp::Var&amp; var_message) {
  if (!var_message.is_string())
    return;

  // Parse message into: instruction file_name_length file_name [file_text]
  std::string message = var_message.AsString();
  std::string instruction;
  std::string file_name;
  std::stringstream reader(message);
  int file_name_length;

  reader &gt;&gt; instruction &gt;&gt; file_name_length;
  file_name.resize(file_name_length);
  reader.ignore(1);  // Eat the delimiter
  reader.read(&amp;file_name[0], file_name_length);

  ...

  // Dispatch the instruction
  if (instruction == kLoadPrefix) {
    file_thread_.message_loop().PostWork(
        callback_factory_.NewCallback(&amp;FileIoInstance::Load, file_name));
  } else if (instruction == kSavePrefix) {
    ...
  }
}
</pre>
</section><section id="saving-a-file">
<h3 id="saving-a-file">Saving a file</h3>
<p><code>FileIoInstance::Save</code> is called when the <code>Save</code> button is pressed. First,
it checks to see that the FileSystem has been successfully opened:</p>
<pre class="prettyprint">
if (!file_system_ready_) {
  ShowErrorMessage(&quot;File system is not open&quot;, PP_ERROR_FAILED);
  return;
}
</pre>
<p>It then creates a <code>pp::FileRef</code> resource with the name of the file. A
<code>FileRef</code> resource is a weak reference to a file in the FileSystem; that is,
a file can still be deleted even if there are outstanding <code>FileRef</code>
resources.</p>
<pre class="prettyprint">
pp::FileRef ref(file_system_, file_name.c_str());
</pre>
<p>Next, a <code>pp::FileIO</code> resource is created and opened. The call to
<code>pp::FileIO::Open</code> passes <code>PP_FILEOPEFLAG_WRITE</code> to open the file for
writing, <code>PP_FILEOPENFLAG_CREATE</code> to create a new file if it doesn&#8217;t already
exist and <code>PP_FILEOPENFLAG_TRUNCATE</code> to clear the file of any previous
content:</p>
<pre class="prettyprint">
pp::FileIO file(this);

int32_t open_result =
    file.Open(ref,
              PP_FILEOPENFLAG_WRITE | PP_FILEOPENFLAG_CREATE |
                  PP_FILEOPENFLAG_TRUNCATE,
              pp::BlockUntilComplete());
if (open_result != PP_OK) {
  ShowErrorMessage(&quot;File open for write failed&quot;, open_result);
  return;
}
</pre>
<p>Now that the file is opened, it is written to in chunks. In an asynchronous
model, this would require writing a separate function, storing the current
state on the free store and a chain of callbacks. Because this function is
called off the main thread, <code>pp::FileIO::Write</code> can be called synchronously
and a conventional do/while loop can be used:</p>
<pre class="prettyprint">
int64_t offset = 0;
int32_t bytes_written = 0;
do {
  bytes_written = file.Write(offset,
                             file_contents.data() + offset,
                             file_contents.length(),
                             pp::BlockUntilComplete());
  if (bytes_written &gt; 0) {
    offset += bytes_written;
  } else {
    ShowErrorMessage(&quot;File write failed&quot;, bytes_written);
    return;
  }
} while (bytes_written &lt; static_cast&lt;int64_t&gt;(file_contents.length()));
</pre>
<p>Finally, the file is flushed to push all changes to disk:</p>
<pre class="prettyprint">
int32_t flush_result = file.Flush(pp::BlockUntilComplete());
if (flush_result != PP_OK) {
  ShowErrorMessage(&quot;File fail to flush&quot;, flush_result);
  return;
}
</pre>
</section><section id="loading-a-file">
<h3 id="loading-a-file">Loading a file</h3>
<p><code>FileIoInstance::Load</code> is called when the <code>Load</code> button is pressed. Like
the <code>Save</code> function, <code>Load</code> first checks to see if the FileSystem has been
successfully opened, and creates a new <code>FileRef</code>:</p>
<pre class="prettyprint">
if (!file_system_ready_) {
  ShowErrorMessage(&quot;File system is not open&quot;, PP_ERROR_FAILED);
  return;
}
pp::FileRef ref(file_system_, file_name.c_str());
</pre>
<p>Next, <code>Load</code> creates and opens a new <code>FileIO</code> resource, passing
<code>PP_FILEOPENFLAG_READ</code> to open the file for reading. The result is compared
to <code>PP_ERROR_FILENOTFOUND</code> to give a better error message when the file
doesn&#8217;t exist:</p>
<pre class="prettyprint">
int32_t open_result =
    file.Open(ref, PP_FILEOPENFLAG_READ, pp::BlockUntilComplete());
if (open_result == PP_ERROR_FILENOTFOUND) {
  ShowErrorMessage(&quot;File not found&quot;, open_result);
  return;
} else if (open_result != PP_OK) {
  ShowErrorMessage(&quot;File open for read failed&quot;, open_result);
  return;
}
</pre>
<p>Then <code>Load</code> calls <code>pp::FileIO::Query</code> to get metadata about the file, such
as its size. This is used to allocate a <code>std::vector</code> buffer that holds the
data from the file in memory:</p>
<pre class="prettyprint">
int32_t query_result = file.Query(&amp;info, pp::BlockUntilComplete());
if (query_result != PP_OK) {
  ShowErrorMessage(&quot;File query failed&quot;, query_result);
  return;
}

...

std::vector&lt;char&gt; data(info.size);
</pre>
<p>Similar to <code>Save</code>, a conventional while loop is used to read the file into
the newly allocated buffer:</p>
<pre class="prettyprint">
int64_t offset = 0;
int32_t bytes_read = 0;
int32_t bytes_to_read = info.size;
while (bytes_to_read &gt; 0) {
  bytes_read = file.Read(offset,
                         &amp;data[offset],
                         data.size() - offset,
                         pp::BlockUntilComplete());
  if (bytes_read &gt; 0) {
    offset += bytes_read;
    bytes_to_read -= bytes_read;
  } else if (bytes_read &lt; 0) {
    // If bytes_read &lt; PP_OK then it indicates the error code.
    ShowErrorMessage(&quot;File read failed&quot;, bytes_read);
    return;
  }
}
</pre>
<p>Finally, the contents of the file are sent back to JavaScript, to be displayed
on the page. This example uses &#8220;<code>DISP|</code>&#8221; as a prefix command for display
information:</p>
<pre class="prettyprint">
std::string string_data(data.begin(), data.end());
PostMessage(&quot;DISP|&quot; + string_data);
ShowStatusMessage(&quot;Load success&quot;);
</pre>
</section><section id="id1">
<h3 id="id1">Deleting a file</h3>
<p><code>FileIoInstance::Delete</code> is called when the <code>Delete</code> button is pressed.
First, it checks whether the FileSystem has been opened, and creates a new
<code>FileRef</code>:</p>
<pre class="prettyprint">
if (!file_system_ready_) {
  ShowErrorMessage(&quot;File system is not open&quot;, PP_ERROR_FAILED);
  return;
}
pp::FileRef ref(file_system_, file_name.c_str());
</pre>
<p>Unlike <code>Save</code> and <code>Load</code>, <code>Delete</code> is called on the <code>FileRef</code> resource,
not a <code>FileIO</code> resource. Note that the result is checked for
<code>PP_ERROR_FILENOTFOUND</code> to give a better error message when trying to delete
a non-existent file:</p>
<pre class="prettyprint">
int32_t result = ref.Delete(pp::BlockUntilComplete());
if (result == PP_ERROR_FILENOTFOUND) {
  ShowStatusMessage(&quot;File/Directory not found&quot;);
  return;
} else if (result != PP_OK) {
  ShowErrorMessage(&quot;Deletion failed&quot;, result);
  return;
}
</pre>
</section><section id="listing-files-in-a-directory">
<h3 id="listing-files-in-a-directory">Listing files in a directory</h3>
<p><code>FileIoInstance::List</code> is called when the <code>List Directory</code> button is
pressed. Like all other operations, it checks whether the FileSystem has been
opened and creates a new <code>FileRef</code>:</p>
<pre class="prettyprint">
if (!file_system_ready_) {
  ShowErrorMessage(&quot;File system is not open&quot;, PP_ERROR_FAILED);
  return;
}

pp::FileRef ref(file_system_, dir_name.c_str());
</pre>
<p>Unlike the other operations, it does not make a blocking call to
<code>pp::FileRef::ReadDirectoryEntries</code>. Since <code>ReadDirectoryEntries</code> returns
the resulting directory entries in its callback, a new callback object is
created pointing to <code>FileIoInstance::ListCallback</code>.</p>
<p>The <code>pp::CompletionCallbackFactory</code> template class is used to instantiate a
new callback. Notice that the <code>FileRef</code> resource is passed as a parameter;
this will add a reference count to the callback object, to keep the <code>FileRef</code>
resource from being destroyed when the function finishes.</p>
<pre class="prettyprint">
// Pass ref along to keep it alive.
ref.ReadDirectoryEntries(callback_factory_.NewCallbackWithOutput(
    &amp;FileIoInstance::ListCallback, ref));
</pre>
<p><code>FileIoInstance::ListCallback</code> then gets the results passed as a
<code>std::vector</code> of <code>pp::DirectoryEntry</code> objects, and sends them to
JavaScript:</p>
<pre class="prettyprint">
void ListCallback(int32_t result,
                  const std::vector&lt;pp::DirectoryEntry&gt;&amp; entries,
                  pp::FileRef /*unused_ref*/) {
  if (result != PP_OK) {
    ShowErrorMessage(&quot;List failed&quot;, result);
    return;
  }

  std::stringstream ss;
  ss &lt;&lt; &quot;LIST&quot;;
  for (size_t i = 0; i &lt; entries.size(); ++i) {
    pp::Var name = entries[i].file_ref().GetName();
    if (name.is_string()) {
      ss &lt;&lt; &quot;|&quot; &lt;&lt; name.AsString();
    }
  }
  PostMessage(ss.str());
  ShowStatusMessage(&quot;List success&quot;);
}
</pre>
</section><section id="making-a-new-directory">
<h3 id="making-a-new-directory">Making a new directory</h3>
<p><code>FileIoInstance::MakeDir</code> is called when the <code>Make Directory</code> button is
pressed. Like all other operations, it checks whether the FileSystem has been
opened and creates a new <code>FileRef</code>:</p>
<pre class="prettyprint">
if (!file_system_ready_) {
  ShowErrorMessage(&quot;File system is not open&quot;, PP_ERROR_FAILED);
  return;
}
pp::FileRef ref(file_system_, dir_name.c_str());
</pre>
<p>Then the <code>pp::FileRef::MakeDirectory</code> function is called.</p>
<pre class="prettyprint">
int32_t result = ref.MakeDirectory(
    PP_MAKEDIRECTORYFLAG_NONE, pp::BlockUntilComplete());
if (result != PP_OK) {
  ShowErrorMessage(&quot;Make directory failed&quot;, result);
  return;
}
ShowStatusMessage(&quot;Make directory success&quot;);
</pre>
</section></section></section>

{{/partials.standard_nacl_article}}