1 .. _devguide-coding-fileio: 2 3 ######## 4 File I/O 5 ######## 6 7 .. contents:: 8 :local: 9 :backlinks: none 10 :depth: 2 11 12 Introduction 13 ============ 14 15 This section describes how to use the `FileIO API 16 </native-client/pepper_stable/cpp/classpp_1_1_file_i_o>`_ to read and write 17 files using a local secure data store. 18 19 You might use the File IO API with the URL Loading APIs to create an overall 20 data download and caching solution for your NaCl applications. For example: 21 22 #. Use the File IO APIs to check the local disk to see if a file exists that 23 your program needs. 24 #. If the file exists locally, load it into memory using the File IO API. If 25 the file doesn't exist locally, use the URL Loading API to retrieve the 26 file from the server. 27 #. Use the File IO API to write the file to disk. 28 #. Load the file into memory using the File IO API when needed by your 29 application. 30 31 The example discussed in this section is included in the SDK in the directory 32 ``examples/api/file_io``. 33 34 Reference information 35 ===================== 36 37 For reference information related to FileIO, see the following documentation: 38 39 * `file_io.h </native-client/pepper_stable/cpp/file__io_8h>`_ - API to create a 40 FileIO object 41 * `file_ref.h </native-client/pepper_stable/cpp/file__ref_8h>`_ - API to create 42 a file reference or "weak pointer" to a file in a file system 43 * `file_system.h </native-client/pepper_stable/cpp/file__system_8h>`_ - API to 44 create a file system associated with a file 45 46 Local file I/O 47 ============== 48 49 Chrome provides an obfuscated, restricted area on disk to which a web app can 50 safely `read and write files 51 <https://developers.google.com/chrome/whitepapers/storage#persistent>`_. The 52 Pepper FileIO, FileRef, and FileSystem APIs (collectively called the File IO 53 APIs) allow you to access this sandboxed local disk so you can read and write 54 files and manage caching yourself. The data is persistent between launches of 55 Chrome, and is not removed unless your application deletes it or the user 56 manually deletes it. There is no limit to the amount of local data you can 57 use, other than the actual space available on the local drive. 58 59 .. _quota_management: 60 .. _enabling_file_access: 61 62 Enabling local file I/O 63 ----------------------- 64 65 The easiest way to enable the writing of persistent local data is to include 66 the `unlimitedStorage permission 67 </extensions/declare_permissions#unlimitedStorage>`_ in your Chrome Web Store 68 manifest file. With this permission you can use the Pepper FileIO API without 69 the need to request disk space at run time. When the user installs the app 70 Chrome displays a message announcing that the app writes to the local disk. 71 72 If you do not use the ``unlimitedStorage`` permission you must include 73 JavaScript code that calls the `HTML5 Quota Management API 74 <http://updates.html5rocks.com/2011/11/Quota-Management-API-Fast-Facts>`_ to 75 explicitly request local disk space before using the FileIO API. In this case 76 Chrome will prompt the user to accept a requestQuota call every time one is 77 made. 78 79 Testing local file I/O 80 ---------------------- 81 82 You should be aware that using the ``unlimitedStorage`` manifest permission 83 constrains the way you can test your app. Three of the four techniques 84 described in :doc:`Running Native Client Applications <../devcycle/running>` 85 read the Chrome Web Store manifest file and enable the ``unlimitedStorage`` 86 permission when it appears, but the first technique (local server) does not. 87 If you want to test the file IO portion of your app with a simple local server, 88 you need to include JavaScript code that calls the HTML5 Quota Management API. 89 When you deliver your application you can replace this code with the 90 ``unlimitedStorage`` manifest permission. 91 92 The ``file_io`` example 93 ======================= 94 95 The Native Client SDK includes an example, ``file_io``, that demonstrates how 96 to read and write a local disk file. Since you will probably run the example 97 from a local server without a Chrome Web Store manifest file, the example's 98 index file uses JavaScript to perform the Quota Management setup as described 99 above. The example has these primary files: 100 101 * ``index.html`` - The HTML code that launches the Native Client module and 102 displays the user interface. 103 * ``example.js`` - JavaScript code that requests quota (as described above). It 104 also listens for user interaction with the user interface, and forwards the 105 requests to the Native Client module. 106 * ``file_io.cc`` - The code that sets up and provides an entry point to the 107 Native Client module. 108 109 The remainder of this section covers the code in the ``file_io.cc`` file for 110 reading and writing files. 111 112 File I/O overview 113 ----------------- 114 115 Like many Pepper APIs, the File IO API includes a set of methods that execute 116 asynchronously and that invoke callback functions in your Native Client module. 117 Unlike most other examples, the ``file_io`` example also demonstrates how to 118 make Pepper calls synchronously on a worker thread. 119 120 It is illegal to make blocking calls to Pepper on the module's main thread. 121 This restriction is lifted when running on a worker thread---this is called 122 "calling Pepper off the main thread". This often simplifies the logic of your 123 code; multiple asynchronous Pepper functions can be called from one function on 124 your worker thread, so you can use the stack and standard control flow 125 structures normally. 126 127 The high-level flow for the ``file_io`` example is described below. Note that 128 methods in the namespace ``pp`` are part of the Pepper C++ API. 129 130 Creating and writing a file 131 --------------------------- 132 133 Following are the high-level steps involved in creating and writing to a 134 file: 135 136 #. ``pp::FileIO::Open`` is called with the ``PP_FILEOPEN_FLAG_CREATE`` flag to 137 create a file. Because the callback function is ``pp::BlockUntilComplete``, 138 this thread is blocked until ``Open`` succeeds or fails. 139 #. ``pp::FileIO::Write`` is called to write the contents. Again, the thread is 140 blocked until the call to ``Write`` completes. If there is more data to 141 write, ``Write`` is called again. 142 #. When there is no more data to write, call ``pp::FileIO::Flush``. 143 144 Opening and reading a file 145 -------------------------- 146 147 Following are the high-level steps involved in opening and reading a file: 148 149 #. ``pp::FileIO::Open`` is called to open the file. Because the callback 150 function is ``pp::BlockUntilComplete``, this thread is blocked until Open 151 succeeds or fails. 152 #. ``pp::FileIO::Query`` is called to query information about the file, such as 153 its file size. The thread is blocked until ``Query`` completes. 154 #. ``pp::FileIO::Read`` is called to read the contents. The thread is blocked 155 until ``Read`` completes. If there is more data to read, ``Read`` is called 156 again. 157 158 Deleting a file 159 --------------- 160 161 Deleting a file is straightforward: call ``pp::FileRef::Delete``. The thread is 162 blocked until ``Delete`` completes. 163 164 Making a directory 165 ------------------ 166 167 Making a directory is also straightforward: call ``pp::File::MakeDirectory``. 168 The thread is blocked until ``MakeDirectory`` completes. 169 170 Listing the contents of a directory 171 ----------------------------------- 172 173 Following are the high-level steps involved in listing a directory: 174 175 #. ``pp::FileRef::ReadDirectoryEntries`` is called, and given a directory entry 176 to list. A callback is given as well; many of the other functions use 177 ``pp::BlockUntilComplete``, but ``ReadDirectoryEntries`` returns results in 178 its callback, so it must be specified. 179 #. When the call to ``ReadDirectoryEntries`` completes, it calls 180 ``ListCallback`` which packages up the results into a string message, and 181 sends it to JavaScript. 182 183 ``file_io`` deep dive 184 ===================== 185 186 The ``file_io`` example displays a user interface with a couple of fields and 187 several buttons. Following is a screenshot of the ``file_io`` example: 188 189 .. image:: /images/fileioexample.png 190 191 Each radio button is a file operation you can perform, with some reasonable 192 default values for filenames. Try typing a message in the large input box and 193 clicking ``Save``, then switching to the ``Load File`` operation, and 194 clicking ``Load``. 195 196 Let's take a look at what is going on under the hood. 197 198 Opening a file system and preparing for file I/O 199 ------------------------------------------------ 200 201 ``pp::Instance::Init`` is called when an instance of a module is created. In 202 this example, ``Init`` starts a new thread (via the ``pp::SimpleThread`` 203 class), and tells it to open the filesystem: 204 205 .. naclcode:: 206 207 virtual bool Init(uint32_t /*argc*/, 208 const char * /*argn*/ [], 209 const char * /*argv*/ []) { 210 file_thread_.Start(); 211 // Open the file system on the file_thread_. Since this is the first 212 // operation we perform there, and because we do everything on the 213 // file_thread_ synchronously, this ensures that the FileSystem is open 214 // before any FileIO operations execute. 215 file_thread_.message_loop().PostWork( 216 callback_factory_.NewCallback(&FileIoInstance::OpenFileSystem)); 217 return true; 218 } 219 220 When the file thread starts running, it will call ``OpenFileSystem``. This 221 calls ``pp::FileSystem::Open`` and blocks the file thread until the function 222 returns. 223 224 .. Note:: 225 :class: note 226 227 Note that the call to ``pp::FileSystem::Open`` uses 228 ``pp::BlockUntilComplete`` as its callback. This is only possible because we 229 are running off the main thread; if you try to make a blocking call from the 230 main thread, the function will return the error 231 ``PP_ERROR_BLOCKS_MAIN_THREAD``. 232 233 .. naclcode:: 234 235 void OpenFileSystem(int32_t /*result*/) { 236 int32_t rv = file_system_.Open(1024 * 1024, pp::BlockUntilComplete()); 237 if (rv == PP_OK) { 238 file_system_ready_ = true; 239 // Notify the user interface that we're ready 240 PostMessage("READY|"); 241 } else { 242 ShowErrorMessage("Failed to open file system", rv); 243 } 244 } 245 246 Handling messages from JavaScript 247 --------------------------------- 248 249 When you click the ``Save`` button, JavaScript posts a message to the NaCl 250 module with the file operation to perform sent as a string (See :doc:`Messaging 251 System <message-system>` for more details on message passing). The string is 252 parsed by ``HandleMessage``, and new work is added to the file thread: 253 254 .. naclcode:: 255 256 virtual void HandleMessage(const pp::Var& var_message) { 257 if (!var_message.is_string()) 258 return; 259 260 // Parse message into: instruction file_name_length file_name [file_text] 261 std::string message = var_message.AsString(); 262 std::string instruction; 263 std::string file_name; 264 std::stringstream reader(message); 265 int file_name_length; 266 267 reader >> instruction >> file_name_length; 268 file_name.resize(file_name_length); 269 reader.ignore(1); // Eat the delimiter 270 reader.read(&file_name[0], file_name_length); 271 272 ... 273 274 // Dispatch the instruction 275 if (instruction == kLoadPrefix) { 276 file_thread_.message_loop().PostWork( 277 callback_factory_.NewCallback(&FileIoInstance::Load, file_name)); 278 } else if (instruction == kSavePrefix) { 279 ... 280 } 281 } 282 283 Saving a file 284 ------------- 285 286 ``FileIoInstance::Save`` is called when the ``Save`` button is pressed. First, 287 it checks to see that the FileSystem has been successfully opened: 288 289 .. naclcode:: 290 291 if (!file_system_ready_) { 292 ShowErrorMessage("File system is not open", PP_ERROR_FAILED); 293 return; 294 } 295 296 It then creates a ``pp::FileRef`` resource with the name of the file. A 297 ``FileRef`` resource is a weak reference to a file in the FileSystem; that is, 298 a file can still be deleted even if there are outstanding ``FileRef`` 299 resources. 300 301 .. naclcode:: 302 303 pp::FileRef ref(file_system_, file_name.c_str()); 304 305 Next, a ``pp::FileIO`` resource is created and opened. The call to 306 ``pp::FileIO::Open`` passes ``PP_FILEOPEFLAG_WRITE`` to open the file for 307 writing, ``PP_FILEOPENFLAG_CREATE`` to create a new file if it doesn't already 308 exist and ``PP_FILEOPENFLAG_TRUNCATE`` to clear the file of any previous 309 content: 310 311 .. naclcode:: 312 313 pp::FileIO file(this); 314 315 int32_t open_result = 316 file.Open(ref, 317 PP_FILEOPENFLAG_WRITE | PP_FILEOPENFLAG_CREATE | 318 PP_FILEOPENFLAG_TRUNCATE, 319 pp::BlockUntilComplete()); 320 if (open_result != PP_OK) { 321 ShowErrorMessage("File open for write failed", open_result); 322 return; 323 } 324 325 Now that the file is opened, it is written to in chunks. In an asynchronous 326 model, this would require writing a separate function, storing the current 327 state on the free store and a chain of callbacks. Because this function is 328 called off the main thread, ``pp::FileIO::Write`` can be called synchronously 329 and a conventional do/while loop can be used: 330 331 .. naclcode:: 332 333 int64_t offset = 0; 334 int32_t bytes_written = 0; 335 do { 336 bytes_written = file.Write(offset, 337 file_contents.data() + offset, 338 file_contents.length(), 339 pp::BlockUntilComplete()); 340 if (bytes_written > 0) { 341 offset += bytes_written; 342 } else { 343 ShowErrorMessage("File write failed", bytes_written); 344 return; 345 } 346 } while (bytes_written < static_cast<int64_t>(file_contents.length())); 347 348 Finally, the file is flushed to push all changes to disk: 349 350 .. naclcode:: 351 352 int32_t flush_result = file.Flush(pp::BlockUntilComplete()); 353 if (flush_result != PP_OK) { 354 ShowErrorMessage("File fail to flush", flush_result); 355 return; 356 } 357 358 Loading a file 359 -------------- 360 361 ``FileIoInstance::Load`` is called when the ``Load`` button is pressed. Like 362 the ``Save`` function, ``Load`` first checks to see if the FileSystem has been 363 successfully opened, and creates a new ``FileRef``: 364 365 .. naclcode:: 366 367 if (!file_system_ready_) { 368 ShowErrorMessage("File system is not open", PP_ERROR_FAILED); 369 return; 370 } 371 pp::FileRef ref(file_system_, file_name.c_str()); 372 373 Next, ``Load`` creates and opens a new ``FileIO`` resource, passing 374 ``PP_FILEOPENFLAG_READ`` to open the file for reading. The result is compared 375 to ``PP_ERROR_FILENOTFOUND`` to give a better error message when the file 376 doesn't exist: 377 378 .. naclcode:: 379 380 int32_t open_result = 381 file.Open(ref, PP_FILEOPENFLAG_READ, pp::BlockUntilComplete()); 382 if (open_result == PP_ERROR_FILENOTFOUND) { 383 ShowErrorMessage("File not found", open_result); 384 return; 385 } else if (open_result != PP_OK) { 386 ShowErrorMessage("File open for read failed", open_result); 387 return; 388 } 389 390 Then ``Load`` calls ``pp::FileIO::Query`` to get metadata about the file, such 391 as its size. This is used to allocate a ``std::vector`` buffer that holds the 392 data from the file in memory: 393 394 .. naclcode:: 395 396 int32_t query_result = file.Query(&info, pp::BlockUntilComplete()); 397 if (query_result != PP_OK) { 398 ShowErrorMessage("File query failed", query_result); 399 return; 400 } 401 402 ... 403 404 std::vector<char> data(info.size); 405 406 Similar to ``Save``, a conventional while loop is used to read the file into 407 the newly allocated buffer: 408 409 .. naclcode:: 410 411 int64_t offset = 0; 412 int32_t bytes_read = 0; 413 int32_t bytes_to_read = info.size; 414 while (bytes_to_read > 0) { 415 bytes_read = file.Read(offset, 416 &data[offset], 417 data.size() - offset, 418 pp::BlockUntilComplete()); 419 if (bytes_read > 0) { 420 offset += bytes_read; 421 bytes_to_read -= bytes_read; 422 } else if (bytes_read < 0) { 423 // If bytes_read < PP_OK then it indicates the error code. 424 ShowErrorMessage("File read failed", bytes_read); 425 return; 426 } 427 } 428 429 Finally, the contents of the file are sent back to JavaScript, to be displayed 430 on the page. This example uses "``DISP|``" as a prefix command for display 431 information: 432 433 .. naclcode:: 434 435 std::string string_data(data.begin(), data.end()); 436 PostMessage("DISP|" + string_data); 437 ShowStatusMessage("Load success"); 438 439 Deleting a file 440 --------------- 441 442 ``FileIoInstance::Delete`` is called when the ``Delete`` button is pressed. 443 First, it checks whether the FileSystem has been opened, and creates a new 444 ``FileRef``: 445 446 .. naclcode:: 447 448 if (!file_system_ready_) { 449 ShowErrorMessage("File system is not open", PP_ERROR_FAILED); 450 return; 451 } 452 pp::FileRef ref(file_system_, file_name.c_str()); 453 454 Unlike ``Save`` and ``Load``, ``Delete`` is called on the ``FileRef`` resource, 455 not a ``FileIO`` resource. Note that the result is checked for 456 ``PP_ERROR_FILENOTFOUND`` to give a better error message when trying to delete 457 a non-existent file: 458 459 .. naclcode:: 460 461 int32_t result = ref.Delete(pp::BlockUntilComplete()); 462 if (result == PP_ERROR_FILENOTFOUND) { 463 ShowStatusMessage("File/Directory not found"); 464 return; 465 } else if (result != PP_OK) { 466 ShowErrorMessage("Deletion failed", result); 467 return; 468 } 469 470 Listing files in a directory 471 ---------------------------- 472 473 ``FileIoInstance::List`` is called when the ``List Directory`` button is 474 pressed. Like all other operations, it checks whether the FileSystem has been 475 opened and creates a new ``FileRef``: 476 477 .. naclcode:: 478 479 if (!file_system_ready_) { 480 ShowErrorMessage("File system is not open", PP_ERROR_FAILED); 481 return; 482 } 483 484 pp::FileRef ref(file_system_, dir_name.c_str()); 485 486 Unlike the other operations, it does not make a blocking call to 487 ``pp::FileRef::ReadDirectoryEntries``. Since ``ReadDirectoryEntries`` returns 488 the resulting directory entries in its callback, a new callback object is 489 created pointing to ``FileIoInstance::ListCallback``. 490 491 The ``pp::CompletionCallbackFactory`` template class is used to instantiate a 492 new callback. Notice that the ``FileRef`` resource is passed as a parameter; 493 this will add a reference count to the callback object, to keep the ``FileRef`` 494 resource from being destroyed when the function finishes. 495 496 .. naclcode:: 497 498 // Pass ref along to keep it alive. 499 ref.ReadDirectoryEntries(callback_factory_.NewCallbackWithOutput( 500 &FileIoInstance::ListCallback, ref)); 501 502 ``FileIoInstance::ListCallback`` then gets the results passed as a 503 ``std::vector`` of ``pp::DirectoryEntry`` objects, and sends them to 504 JavaScript: 505 506 .. naclcode:: 507 508 void ListCallback(int32_t result, 509 const std::vector<pp::DirectoryEntry>& entries, 510 pp::FileRef /*unused_ref*/) { 511 if (result != PP_OK) { 512 ShowErrorMessage("List failed", result); 513 return; 514 } 515 516 std::stringstream ss; 517 ss << "LIST"; 518 for (size_t i = 0; i < entries.size(); ++i) { 519 pp::Var name = entries[i].file_ref().GetName(); 520 if (name.is_string()) { 521 ss << "|" << name.AsString(); 522 } 523 } 524 PostMessage(ss.str()); 525 ShowStatusMessage("List success"); 526 } 527 528 Making a new directory 529 ---------------------- 530 531 ``FileIoInstance::MakeDir`` is called when the ``Make Directory`` button is 532 pressed. Like all other operations, it checks whether the FileSystem has been 533 opened and creates a new ``FileRef``: 534 535 .. naclcode:: 536 537 if (!file_system_ready_) { 538 ShowErrorMessage("File system is not open", PP_ERROR_FAILED); 539 return; 540 } 541 pp::FileRef ref(file_system_, dir_name.c_str()); 542 543 Then the ``pp::FileRef::MakeDirectory`` function is called. 544 545 .. naclcode:: 546 547 int32_t result = ref.MakeDirectory( 548 PP_MAKEDIRECTORYFLAG_NONE, pp::BlockUntilComplete()); 549 if (result != PP_OK) { 550 ShowErrorMessage("Make directory failed", result); 551 return; 552 } 553 ShowStatusMessage("Make directory success"); 554