Multi-part Download with Multiple Threads

This topic shows you how to use multiple threads to connect to an FTP server and download a file. First, we need to define a class which contains information about the file, and the threads. In this example, it's named GlobalInfo. And then we define a class for Ftp Threads, it's named FtpThread. Finally, in the main routine, we create and start 3 threads to download the parts of the file. As a result, the download speed is improved if the file is large. We take advantage of the GetDownloadStream method with the following example code:

using System;
using System.Collections.Generic;
using System.Text;
using System.IO;
using Atp.Net;
using System.Threading;

namespace Samples
{
    class GlobalInfo
    {
        // Maintain some information about the threads.
        public int CompletedThreads;
        public int Threads;
        public ManualResetEvent Event;
        public byte[][] Streams;
        public long Length = -1;
        public int BlockSize;
        public bool Finish;
    }

    class FtpThread
    {
        int _offset;
        int _blockSize;
        GlobalInfo _info;
        Ftp _client;
        int _index;
        
        public FtpThread(GlobalInfo info, int index)
        {
            _info = info;
            _index = index;
        }

        public void Start()
        {
            const string remoteFile = "/pub/test.zip";
            _client = new Ftp();
            _client.Connect("atp-inc.net");
            _client.Authenticate("test", "test");

            byte[] buf;

            lock (_info)
            {
                if (_info.Length == -1)
                {
                    // Get file length.
                    _info.Length = _client.GetFileLength(remoteFile);
                    // Calculate block size.
                    _info.BlockSize = (int)(_info.Length / _info.Threads);
                    _info.Streams = new byte[_info.Threads][];
                }

                // Calculate offset.
                _offset = _info.BlockSize * _index;
                _blockSize = (int)((_index == _info.Threads - 1) ? (_info.Length - _index * _info.BlockSize) : _info.BlockSize);

                buf = new byte[_blockSize];
                _info.Streams[_index] = buf;            
            }

            // Get download stream.
            Stream downloader = _client.GetDownloadStream(remoteFile, SeekOrigin.Begin, _offset);
            int count = 0;
            // Download data.
            while (true)
            {
                int size = (count + 4048 > _blockSize) ? (_blockSize - count) : 4048;
                int read = downloader.Read(buf, count, size);
                if (count + read >= _blockSize || read == 0)
                    break;
                count += read;
            } 
            downloader.Close();            

            _client.Disconnect();

            lock (_info)
            {
                _info.CompletedThreads++;
                if (_info.CompletedThreads == _info.Threads && !_info.Finish)
                {
                    _info.Event.Set();
                    // All threads have completed.

                    // Combine
                    using (FileStream sw = System.IO.File.OpenWrite(@"c:\temp\test.zip"))
                    {
                        foreach (byte[] s in _info.Streams)
                        {
                            sw.Write(s, 0, (int)s.Length);
                        }
                    }

                    _info.Finish = true;
                }
            }
        }
    }

    class MultiParts
    {
        static ManualResetEvent _event = new ManualResetEvent(false);
        static Ftp client;

        static void Main()
        {
            const int threads = 3;

            // Use 3 threads to download a remote file to streams and then save the streams into a local file.
            int offset = 0;
            
            GlobalInfo ginfo = new GlobalInfo();
            ginfo.Threads = threads;
            ginfo.Event = _event;

            for (int i = 0; i < threads; i++)
            {
                FtpThread fthread = new FtpThread(ginfo, i);
                Thread th = new Thread(fthread.Start);
                th.Start();
            }
            _event.WaitOne();
        }
    }
}

Click here to download the Ultimate FTP Component for .NET, or here to download the .NET CF version.

Ultimate FTP in Microsoft Windows PowerShell

ATP Inc would like to illustrate how to use Ultimate FTP in PowerShell in this topic. PowerShell is the new command line and scripting language developed by Microsoft to helps IT professionals complete their tasks more efficiently with greater control and productivity on Microsoft Windows Platforms. The PowerShell contains more than 130 command line tools especially designed for administrators.

Windows PowerShell is built on top of the .NET Framework, so it gives administrators a wide-range of methods and extensions. With Ultimate FTP, now you can upload and download files to FTP server easily with a few lines of code. Since PowerShell accepts and returns .NET objects, FTP classes are accessible to the PowerShell, in which you can use the component to build a complex application like a C# or VB.NET application. This component can be downloaded from our website.

To get start, we begin with uploading several files with a single line of code:

Unlike many other FTP components, Ultimate FTP enables you to upload and download multiple files with different extensions with a single line of code. We take advantages of this feature with the following PowerShell sample:

# Load DLL
 Add-Type -Path C:\InstalledDir\UltimateFtp.dll
 # Set source path and files to upload
 $sourcePath = "C:\Temp\*.ps1;*.dat;*.zip"
 # Set destination path
 $destinationPath = "/my dir"
 # Create a new instance of the Ftp class
 $ftp = New-Object Atp.Net.Ftp
 $ftp.Connect("myserver")
 $ftp.Authenticate("user", "pass")
 # Upload files
 $ftp.UploadFiles($sourcePath, $destinationPath)
 
 $ftp.Close()
 $ftp.Dispose()

Now we will download files from the FTP server to the local disk:

We gives you the ease-of-use of the FTP component not only in the upload methods, but also in the download methods. The following PowerShell code example demonstrates how to load the UltimateFtp.DLL assembly and download files from the FTP server:

# Load DLL
Add-Type -Path C:\InstalledDir\UltimateFtp.dll
# Set source path and files to upload
$sourcePath = "/my dir/*.ps1;*.dat;*.zip"
# Set destination path
$destinationPath = "C:\Temp2"
# Create a new instance of the Ftp class
$ftp = New-Object Atp.Net.Ftp
 $ftp.Connect("myserver")
 $ftp.Authenticate("user", "pass")
 # Download files
$ftp.DownloadFiles($sourcePath, $destinationPath)

$ftp.Close()
$ftp.Dispose()

Now you see with Ultimate FTP Component and a little knowledge of PowerShell language, you can build a comprehensive file transfer application running on the Windows PowerShell.

Click here to download the Ultimate FTP Component for .NET, or here to download the .NET CF version.

Listing a directory Asynchronously

In UltimateFtp, you can use BeginListDirectory, BeginListName or BeginListRawName methods of the Ftp class to asynchronously retrieve the list of files and directories in the specified directory on the FTP server. These methods retrieve the list asynchronously with execution occurring on a new thread, therefore it allows your next line of code to execute  immediately. The event ListDirectoryCompleted, ListNameCompleted or ListRawNameCompleted of the Ftp class under namespace Atp.Net is raised when the BeginListDirectory, BeginListName or BeginListRawName is complete. In the handler method of the ListDirectoryCompleted, ListNameCompleted or ListRawNameCompleted event, you need to call the EndListDirectory, EndListName or EndListRawName method to finish the asynchronous operation.

The following image shows how the FTP Client Demo sample project shows contents of an FTP directory:

FTP Client Demo

To retrieve the list of files and directories in a directory on the FTP server asynchronously, you can simply perform the following steps

  1. Add the FTP component to your application. See Creating a WinForms Application or Creating a Web Application for more details.
  2. Add using directives to your code to create aliases for existing namespaces and avoid having to type the fully qualified type names.

  3. Create a new instance of the Ftp class.
  4. Now you can connect to the FTP server with Connect or BeginConnect methods. The code looks similar to the following:
    C#  
    // Create a new instance.
    Ftp client = new Ftp();
    // Connect to the FTP server.
    client.Connect("myserver");
    // Or you can specify the FTP port with
    // client.Connect("myserver", 21);
    VB.NET  
    ' Create a new instance.
    Dim client As New Ftp()
    ' Connect to the FTP server.
    client.Connect("myserver")
    ' Or you can specify the FTP port with
    ' client.Connect("myserver", 21);

  5. Now you can call the BeginListDirectory to asynchronously retrieve the list of files and directories in the specified directory on the FTP server. Prior to calling BeginListDirectory method, you have to register an event handler to the ListDirectoryCompleted event (you do not need to do that before each call to the BeginListDirectory method, just before the first call). Upon completion of the operation, Ultimate FTP will raise the ListDirectoryCompleted event. When the event is raised, access information contained in the AsyncMethodCompletedEventArgs object. The code looks similar to the following:
    C#  
    // Register an event handler.
    client.ListDirectoryCompleted += client_ListDirectoryCompleted;
    // Get information of all files and directories in '/' remote dir.
    client.BeginListDirectory("/");
    VB.NET  
    ' Register an event handler.
    AddHandler client.ListDirectoryCompleted, AddressOf client_ListDirectoryCompleted
    ' Get information of all files and directories in '/' remote dir.
    client.BeginListDirectory("/")
  6. Now you need to write the code for client_ListDirectoryCompleted event handler. And in the client_ListDirectoryCompleted event handler, write your own code to do something like displaying the received list... The code looks similar to the following:
    C#  
    void client_ListDirectoryCompleted(object sender, AsyncMethodCompletedEventArgs e)
    {
       Ftp client = (Ftp)sender;
       
    try
       {
           
    // Get information of all files and directories in '/' remote dir.
           
    foreach (FtpFileInfo info in client.EndListDirectory(e.AsyncResult))
           {
               Console.WriteLine(
    "Name: {0}, UserId: {1}, Permissions: {2}", info.Name, info.UserId, info.Permissions);
           }
       }
       
    catch (Exception exc)
       {
           Console.WriteLine(
    "Error: " + exc.ToString());
       }
    }
    VB.NET  
    Private Sub client_ListDirectoryCompleted(ByVal sender As Object, ByVal e As AsyncMethodCompletedEventArgs)
        Dim client As Ftp = CType(sender, Ftp)
        Try
            ' Get information of all files and directories in '/' remote dir.
            For Each info As FtpFileInfo In client.EndListDirectory(e.AsyncResult)
                Console.WriteLine("Name: {0}, UserId: {1}, Permissions: {2}", info.Name, info.UserId, info.Permissions)
            Next info
        Catch exc As Exception
            Console.WriteLine("Error: " & exc.ToString())
        End Try
    End Sub
  7. After completing your work, call the Disconnect method to close the FTP session. 

Final Example Code using Ultimate FTP

C#  
public void DoAsyncListDirectory()
{
   
// Create a new instance.
   
Ftp client = new Ftp();
   
// Connect to the FTP server. (i.e. yourdomain.com)
   
client.Connect("myserver");
   
// Authenticate.
   
client.Authenticate("test", "test");
   
// ...
   
// Register an event handler.
   
client.ListDirectoryCompleted += client_ListDirectoryCompleted;
   
// Get information of all files and directories in '/' remote dir.
   
client.BeginListDirectory("/");
   
// ...
   
// Disconnect.
   
client.Disconnect();
}
void client_ListDirectoryCompleted(object sender, AsyncMethodCompletedEventArgs e)
{
   Ftp client = (Ftp)sender;
   
try
   {
       
// Get information of all files and directories in '/' remote dir.
       
foreach (FtpFileInfo info in client.EndListDirectory(e.AsyncResult))
       {
           Console.WriteLine(
"Name: {0}, UserId: {1}, Permissions: {2}", info.Name, info.UserId, info.Permissions);
       }
   }
   
catch (Exception exc)
   {
       Console.WriteLine(
"Error: " + exc.ToString());
   }
}

VB.NET  
Public Sub DoAsyncListDirectory()
    ' Create a new instance.
    Dim client As New Ftp()
    ' Connect to the FTP server.
    client.Connect("atp-inc.net")
    ' Authenticate.
    client.Authenticate("test", "test")
    ' ...
    ' Register an event handler.
    AddHandler client.ListDirectoryCompleted, AddressOf client_ListDirectoryCompleted
    ' Get information of all files and directories in '/' remote dir.
    client.BeginListDirectory("/")
    ' ...
    ' Disconnect.
    client.Disconnect()
End Sub
Private Sub client_ListDirectoryCompleted(ByVal sender As Object, ByVal e As AsyncMethodCompletedEventArgs)
    Dim client As Ftp = CType(sender, Ftp)
    Try
        ' Get information of all files and directories in '/' remote dir.
        For Each info As FtpFileInfo In client.EndListDirectory(e.AsyncResult)
            Console.WriteLine("Name: {0}, UserId: {1}, Permissions: {2}", info.Name, info.UserId, info.Permissions)
        Next info
    Catch exc As Exception
        Console.WriteLine("Error: " & exc.ToString())
    End Try
End Sub

After listing a directory successfully, you can either use the asynchronous methods or synchronous methods to transfer files. For more details on transferring files, see this Transferring multiple files using Ultimate FTP topic.

Click here to download the Ultimate FTP Component for .NET, or here to download the .NET CF version.