Node JS Stream Example

Node JS Stream is an interface. It has been implemented in a lot of node modules such as the fs module, http module, etc. This example will show you how to use node stream to read data from a file, write data to file, and use pipe stream to transfer data from one stream to another.

1. Node Stream Read Data From File.

1. Below is the node stream read data from the file example source code.

   // Require node fs module.
   var file_system = require("fs");
   
   // data variable is used to save read file data.
   var data = '';
   
   var file_path = '/Users/zhaosong/Documents/test.txt';
   
   // Create file read stream.
   var read_stream = file_system.createReadStream(file_path);
   
   // Set read stream encoding to utf8.
   read_stream.setEncoding('utf8');
   
   // Process read stream events.
   // Data reading event.
   read_stream.on('data', function(chunk) {
       data += chunk;
   });
   
   // Read stream complete event.
   read_stream.on('end',function(){
       // Output read data to console.
       console.log("Stream data is : " + data);
   });
   
   // Read stream error event.
   read_stream.on('error', function(err){
       console.log(err.stack);
   });
   
   console.log("Read file stream complete");

2. Below is the above source code output.

   Read file stream complete
   Stream data is : This is a text file.

2. Node Stream Write Data To File.

1. Below is the node stream write data to the file example source code.

   // Require fs module.
   var file_system = require("fs");
   
   // Data which will be wrote to text file.
   var data = 'dev2qa.com is a wonderful website.';
   
   // Data will be wrote to below file.
   var file_path = '/Users/zhaosong/Documents/output.txt';
   
   // Create a write stream object.
   var write_stream = file_system.createWriteStream(file_path);
   
   // Write text with urf-8 character encoding.
   write_stream.write(data,'utf-8');
   
   // Write operation complete.
   write_stream.end();
   
   // Process write stream events.
   // Process write finish event.
   write_stream.on('finish', function() {
       console.log("Write success.");
   });
   
   // Process error event.
   write_stream.on('error', function(err){
       console.log(err.stack);
   });
   
   console.log("Write stream to file complete.");

2. Below is the above source code output.

   Write stream to file complete.
   Write success.

3. Node JS PIPE Stream To Transfer Data Between Input Stream And Output Stream.

1. A pipe stream is used to transfer data from one input stream to another output stream. Below is the pipe stream diagram.
   
2. Below is the example source code that uses node js pipe stream to transfer data from the input stream to the output stream.

   // Require fs module.
   var file_system = require("fs");
   
   var source_file_path = '/Users/zhaosong/Documents/test.txt';
   
   // Create a read stream.
   var read_stream = file_system.createReadStream(source_file_path);
   
   // Process read stream error event.
   read_stream.on('error', function(error){
      console.log(error.stack);
   });
   
   // Create a write stream to overwrite the data in the target file.
   var write_stream = file_system.createWriteStream('target.txt');
   
   // Process write stream finish event.
   write_stream.on('finish', function() {
       console.log("Write success.");
   });
   
   // Process write stream error event.
   write_stream.on('error', function (error) {
      console.log(error.stack);
   });
   
   // If you want to append input stream data to the output stream
   // then set 'flags' to 'a' when create the write stream.
   //var write_stream = file_system.createWriteStream('target.txt', { 'flags': 'a' });
   
   // Use pipe stream to read data from source file through input stream and
   // write those data to destination file through write stream.
   read_stream.pipe(write_stream);
   
   console.log("Data transfer complete.");

3. Below is the above source code executing output.

   Data transfer complete.
   Write success.

4. You can find the file target.txt in the current folder where you execute the js code.

4. Use Pipe Stream To Compress File To Zip Format.

1. This example will compress the target.txt file to zip format file target.txt.gz.
   
2. Below is the example source code that uses node js pipe stream to compress file to zip format file.

   // Require fs module.
   var file_system = require("fs");
   
   // Require zlib module.
   var zlib = require('zlib');
   
   // Open read stream from source file.
   var read_stream = file_system.createReadStream('target.txt');
   
   // Create a zip write stream.
   var zip_write_stream = zlib.createGzip();
   
   // Create pipe stream between the source file read stream and the zip write stream.
   // This pipe stream will compress the source data to zip format.
   var zip_pipe_stream = read_stream.pipe(zip_write_stream);
   
   // Create a write stream to output zip file.
   var write_stream = file_system.createWriteStream('target.txt.gz');
   
   // Create a pipe stream between the zip_pipe_stream and write_stream
   // to write compressed data to target zip file.
   zip_pipe_stream.pipe(write_stream);
   
   console.log("Compress zip file success.");

3. Below is the above example source code executing output.

   Compress zip file success.

5. Use Pipe Stream To UnCompress Zip File.

1. The below source code uses the node js pipe stream module to uncompress the zip file.

   // Require fs module.
   var file_system = require("fs");
   
   // Require zlib module.
   var zlib = require('zlib');
   
   // Create a read strem to read data from source zip file.
   var zip_read_stream = file_system.createReadStream('target.txt.gz');
   
   // Create a zip uncompress stream.
   var unzip_stream = zlib.createGunzip();
   
   // Create a pipe stream to connect zip file read stream and unzip stream.
   var unzip_pipe_stream = zip_read_stream.pipe(unzip_stream);
   
   // Create a target write stream that is used to write data to target file.
   var target_write_stream = file_system.createWriteStream('target1.txt');
   
   // Create a pipe stream to connect unzip pipe stream to target write stream.
   unzip_pipe_stream.pipe(target_write_stream);
   
   console.log("Unzip file success.");

2. Below is the above example source code executing output.

   Unzip file success.

6. Questions & Answers.

6.1 How To Use Node.JS To Get Stream Object From String Object.

6.1.1 Question1.

1. In my Node.js program, I have a string object and I want to create a Stream object from the string object, how can I do it? Thanks.

6.1.2 Answer1.

1. You can use the Node.js streams API to implement it, it is very easy. Below is an example source code.

   // import the Node.js stream module Readable class.
   const Readable = require('stream').Readable;
   
   // create an instance of the Readable class.
   const s = new Readable();
   
   // set the stream object's _read attribute to avoid crash. 
   s._read = () => {}; 
   
   // push the string to the stream object.
   s.push('your string object here');
   s.push(null);

6.1.3 Answer2.

1. You can use the stream.Readable class’s method from to convert an iterable object to a stream object.
2. This method is added from Node.js version 10.17. Because the string object is an iterable object, so you can get streams from the string object with the method from. Below is an example.

   // first import the stream module.
   const { Readable } = require("stream")
   
   // get the stream object from a string object, it will return a character in the string each time.
   const readable = Readable.from("your input string")
   
   
   // get the stream object from a string array, it will return a string from the array each time.
   const readable = Readable.from(["input string"])
   
   // it will execute the below code when the from method is called
   readable.on("data", (chunk) => {
     console.log(chunk)
   })