ComponentSoft.net Ultimate FTP

How to use FTP Component in C#, VB.NET and ASP.NET

Multi-part Download with Multiple Threads

This topic shows you how to use multiple threads to connect to an FTP server and download a file. First, we need to define a class which contains information about the file, and the threads. In this example, it's named GlobalInfo. And then we define a class for Ftp Threads, it's named FtpThread. Finally, in the main routine, we create and start 3 threads to download the parts of the file. As a result, the download speed is improved if the file is large. We take advantage of the GetDownloadStream method with the following example code:

using System;
using System.Collections.Generic;
using System.Text;
using System.IO;
using Atp.Net;
using System.Threading;

namespace Samples
{
    class GlobalInfo
    {
        // Maintain some information about the threads.
        public int CompletedThreads;
        public int Threads;
        public ManualResetEvent Event;
        public byte[][] Streams;
        public long Length = -1;
        public int BlockSize;
        public bool Finish;
    }

    class FtpThread
    {
        int _offset;
        int _blockSize;
        GlobalInfo _info;
        Ftp _client;
        int _index;
        
        public FtpThread(GlobalInfo info, int index)
        {
            _info = info;
            _index = index;
        }

        public void Start()
        {
            const string remoteFile = "/pub/test.zip";
            _client = new Ftp();
            _client.Connect("atp-inc.net");
            _client.Authenticate("test", "test");

            byte[] buf;

            lock (_info)
            {
                if (_info.Length == -1)
                {
                    // Get file length.
                    _info.Length = _client.GetFileLength(remoteFile);
                    // Calculate block size.
                    _info.BlockSize = (int)(_info.Length / _info.Threads);
                    _info.Streams = new byte[_info.Threads][];
                }

                // Calculate offset.
                _offset = _info.BlockSize * _index;
                _blockSize = (int)((_index == _info.Threads - 1) ? (_info.Length - _index * _info.BlockSize) : _info.BlockSize);

                buf = new byte[_blockSize];
                _info.Streams[_index] = buf;            
            }

            // Get download stream.
            Stream downloader = _client.GetDownloadStream(remoteFile, SeekOrigin.Begin, _offset);
            int count = 0;
            // Download data.
            while (true)
            {
                int size = (count + 4048 > _blockSize) ? (_blockSize - count) : 4048;
                int read = downloader.Read(buf, count, size);
                if (count + read >= _blockSize || read == 0)
                    break;
                count += read;
            } 
            downloader.Close();            

            _client.Disconnect();

            lock (_info)
            {
                _info.CompletedThreads++;
                if (_info.CompletedThreads == _info.Threads && !_info.Finish)
                {
                    _info.Event.Set();
                    // All threads have completed.

                    // Combine
                    using (FileStream sw = System.IO.File.OpenWrite(@"c:\temp\test.zip"))
                    {
                        foreach (byte[] s in _info.Streams)
                        {
                            sw.Write(s, 0, (int)s.Length);
                        }
                    }

                    _info.Finish = true;
                }
            }
        }
    }

    class MultiParts
    {
        static ManualResetEvent _event = new ManualResetEvent(false);
        static Ftp client;

        static void Main()
        {
            const int threads = 3;

            // Use 3 threads to download a remote file to streams and then save the streams into a local file.
            int offset = 0;
            
            GlobalInfo ginfo = new GlobalInfo();
            ginfo.Threads = threads;
            ginfo.Event = _event;

            for (int i = 0; i < threads; i++)
            {
                FtpThread fthread = new FtpThread(ginfo, i);
                Thread th = new Thread(fthread.Start);
                th.Start();
            }
            _event.WaitOne();
        }
    }
}

Click here to download the Ultimate FTP Component for .NET, or here to download the .NET CF version.

Downloading multiple files and directories

Download multiple files and directories with ATP UltimateFtp is really simple. You only need to call the DownloadFiles method with few parameters. The component will do the rest of hard work for you. It will loop through the specified directory, find files that match the specified search criteria, create threads, and start downloading. The steps belows show you how to use Ultimate FTP to download files with three threads.

Downloading files using multiple threads

Add using directives to your code to create aliases for existing namespaces and avoid having to type the fully qualified type names. The code looks similar to the following:

C# Copy Code
using Atp.Net;
using Atp.IO;
VB.NET Copy Code
Imports Atp.Net
Imports Atp.IO

Create a new instance of the Ftp class.
C# Copy Code
// Create a new instance.
Ftp client = new Ftp();
VB.NET Copy Code
' Create a new instance.
Dim client As New Ftp()
Register event handlers to the ThreadCompleted and ThreadsCompleted events to get informed when a thread has completed and all threads have completed. The code looks similar to the following:
C# Copy Code
client.ThreadCompleted += client_ThreadCompleted;
client.ThreadsCompleted += client_ThreadsCompleted;
VB.NET Copy Code
AddHandler client.ThreadCompleted, AddressOf client_ThreadCompleted
AddHandler client.ThreadsCompleted, AddressOf client_ThreadsCompleted

Now pass all needed parameters to the DownloadFiles method. The code looks similar to the following:
C# Copy Code
// Download files and subdirectories from "/my folder" to "c:\\my folder" using 3 threads. This waits untils these threads complete.
client.DownloadFiles("/my folder", "c:\\my folder", 3, true);
VB.NET Copy Code
' Download files and subdirectories from "/my folder" to "c:\\my folder" using 3 threads. This waits untils these threads complete.
client.DownloadFiles("/my folder", "c:\my folder", 3, True)

Final example code

C# Copy Code
public void DoMultiThreadsDownloadFiles()
{
   // Create a new instance of the Atp.Net.Ftp class.
   Ftp client = new Ftp();
   // Connect to the server.
   client.Connect("server");
   // Authenticate.
   client.Authenticate("user", "pass");
   client.CommandResponse += client_ResponseRead;
   client.ThreadCompleted += client_ThreadCompleted;
   client.ThreadsCompleted += client_ThreadsCompleted;
   // ...
   // Download files and subdirectories from "/my folder" to "c:\\my folder" using 3 threads. This waits untils these threads complete.
   client.DownloadFiles("/my folder", "c:\\my folder", 3, true);
   // ...
   client.Disconnect();
}
void client_ThreadsCompleted(object sender, ThreadsCompletedEventArgs e)
{
   Console.WriteLine("Multi-threads file transfer completed");
}
void client_ThreadCompleted(object sender, ThreadCompletedEventArgs e)
{
   Console.WriteLine(string.Format("Thread ID {0} completed", e.FileSystem.ThreadId));
}
void client_ResponseRead(object sender, CommandResponseEventArgs e)
{
   Ftp client = (Ftp)sender;
   if (client.ThreadId >= 0)
       if (e.Command != null)
           Console.WriteLine("Thread: {0} - CMD>       {1}", client.ThreadId,
               e.Command);
       else
           Console.WriteLine("Thread: {0} - RESPONSE>  {1}", client.ThreadId,
               e.Response);
}
VB.NET Copy Code
Public Sub DoMultiThreadsDownloadFiles()
    ' Create a new instance.
    Dim client As New Ftp()
    ' Connect to the server.
    client.Connect("server")
    ' Authenticate.
    client.Authenticate("user", "pass")
    AddHandler client.CommandResponse, AddressOf client_ResponseRead
    AddHandler client.ThreadCompleted, AddressOf client_ThreadCompleted
    AddHandler client.ThreadsCompleted, AddressOf client_ThreadsCompleted
    ' ...
    ' Download files and subdirectories from "/my folder" to "c:\\my folder" using 3 threads. This waits untils these threads complete.
    client.DownloadFiles("/my folder", "c:\my folder", 3, True)
    ' ...
    client.Disconnect()
End Sub
Private Sub client_ThreadsCompleted(ByVal sender As Object, ByVal e AsThreadsCompletedEventArgs)
    Console.WriteLine("Multi-threads file transfer completed")
End Sub
Private Sub client_ThreadCompleted(ByVal sender As Object, ByVal e AsThreadCompletedEventArgs)
    Console.WriteLine(String.Format("Thread ID {0} completed", e.FileSystem.ThreadId))
End Sub
Private Sub client_ResponseRead(ByVal sender As Object, ByVal e AsCommandResponseEventArgs)
    Dim client As Ftp = CType(sender, Ftp)
    If client.ThreadId >= 0 Then
        If e.Command IsNot Nothing Then
            Console.WriteLine("Thread: {0} - CMD> {1}", client.ThreadId, e.Command)
        Else
            Console.WriteLine("Thread: {0} - RESPONSE> {1}", client.ThreadId, e.Response)
        End If
    End If
End Sub

You may want to see other topics:

Uploading selected files and directories

 

Click here to download the Ultimate FTP Component for .NET, or here to download the .NET CF version.