DataCollection
仮想化
DataCollection の操作 > 仮想化

Data Virtualization enables efficient processing of data by allowing large amounts of data to be loaded in lesser period of time. C1DataCollection enables data virtualization for larger data sets with enhanced performance. It provides a powerful feature, known as On-demand or Incremental Loading that helps data to be fetched and loaded in chunks as the user scrolls down a list in real time.

loading

In most scenarios, the data comes from different remote source systems. This impacts the time required to show the data, as well as brings an inherent cost in terms of network usage. In order to solve this problem, the data is divided and sent to the client in chunks or pages. There are two widely adopted approaches for data virtualization, pagination and cursor. In Pagination approach, a limit and offset parameter are send to the remote source to specify which portion of the data is requested. This approach returns the total amount of items available, so that the client may know how to retrieve the rest of the items. While, in Cursor approach, a token is send initially, which is null at first, to fetch the pages sequentially. In this case, the received page contains the token to the next one.

C1DataCollection namespace provides base classes such as C1VirtualDataCollection and C1CursorDataCollection to implement pagination and cursor approach respectively. Both classes have an abstract GetPageAsync() method that must be implemented to return the items that will populate the collection. The base classes take the responsibility of caching the data as well as dispatching the requests of the pages according to a series of parameters that prevents too many pages to be requested together.

The GetPageAsync method of C1VirtualDataCollection returns the items in the page as well as a token to the next page. This method uses pageIndex, startingIndex, count, sortDescriptions, filterExpression and cancellation Token as parameters. The parameters pageIndex denotes the index of the requesting page, startingIndex denotes the index where the returned items will be inserted and count denotes the number of items to be returned. The GetPageAsync method returns total number of items (TotalCount) along with a list of the requested number of items (count) starting from an index (startingIndex). To know in detail about the parameters of the method, refer this topic.

The following code snippet depicts the use of C1VirtualDataCollection class:

This is the C# code snippet for WinForms application:

C#
コードのコピー 
public class VirtualModeCollectionView : C1VirtualDataCollection<Customer> 
   {   
        public int TotalCount  { get; set; } = 1_000;
    protected override async Task<Tuple<int, IReadOnlyList<Customer>>> GetPageAsync(int pageIndex, int startingIndex, int count, IReadOnlyList<SortDescription> sortDescriptions = null, FilterExpression filterExpression = null, CancellationToken cancellationToken = default(CancellationToken))
    {
        await Task.Delay(500, cancellationToken);//ネットワークトラフィックをシミュレートします。    
        return new Tuple<int, IReadOnlyList<Customer>>(TotalCount, Enumerable.Range(startingIndex, count).Select(i => new Customer(i)).ToList());
    }
   }

This is the VB code snippet for WinForms application:

VB
コードのコピー 
Public Class VirtualModeCollectionView
    Inherits C1VirtualDataCollection(Of Customer)
    Public Property TotalCount As Integer = 1_000

    Protected Overrides Async Function GetPageAsync(ByVal pageIndex As Integer, ByVal startingIndex As Integer, ByVal count As Integer, ByVal Optional sortDescriptions As IReadOnlyList(Of SortDescription) = Nothing, ByVal Optional filterExpression As FilterExpression = Nothing, ByVal Optional cancellationToken As CancellationToken = Nothing) As Task(Of Tuple(Of Integer, IReadOnlyList(Of Customer)))
        Await Task.Delay(500, cancellationToken)
        Return New Tuple(Of Integer, IReadOnlyList(Of Customer))(TotalCount, Enumerable.Range(startingIndex, count).[Select](Function(i) New Customer(i)).ToList())
    End Function

End Class

This is the C# code snippet for WPF application:

C#
コードのコピー 
public class VirtualModeDataCollection : C1VirtualDataCollection<Customer>
{
    public int TotalCount { get; set; } = 1_000;

    protected override async Task<Tuple<int, IReadOnlyList<Customer>>> GetPageAsync(int pageIndex, int startingIndex, int count, IReadOnlyList<SortDescription> sortDescriptions = null, FilterExpression filterExpression = null, CancellationToken cancellationToken = default(CancellationToken))
    {
        await Task.Delay(500, cancellationToken);//Simulates network traffic.
        return new Tuple<int, IReadOnlyList<Customer>>(TotalCount, Enumerable.Range(startingIndex, count).Select(i => new Customer(i)).ToList());
    }
}

This is the VB code snippet for WPF application:

VB
コードのコピー 
Public Class VirtualModeDataCollection
    Inherits C1VirtualDataCollection(Of Customer)
    Public Property TotalCount As Integer = 1_000
    Protected Overrides Async Function GetPageAsync(pageIndex As Integer, startingIndex As Integer, count As Integer, Optional sortDescriptions As IReadOnlyList(Of SortDescription) = Nothing, Optional filterExpression As FilterExpression = Nothing, Optional cancellationToken As CancellationToken = Nothing) As Task(Of Tuple(Of Integer, IReadOnlyList(Of Customer)))
        Await Task.Delay(500, cancellationToken)
        Return New Tuple(Of Integer, IReadOnlyList(Of Customer))(TotalCount, Enumerable.Range(startingIndex, count).[Select](Function(i) New Customer(i)).ToList())
    End Function
End Class

This is the code snippet for Xamarin Forms:

Xamarin Forms
コードのコピー 
public class VirtualModeDataCollection : C1VirtualDataCollection<Customer>
{
    public int TotalCount { get; set; } = 1_000;

    protected override async Task<Tuple<int, IReadOnlyList<Customer>>> GetPageAsync(int pageIndex, int startingIndex, int count, IReadOnlyList<SortDescription> sortDescriptions = null, FilterExpression filterExpression = null, CancellationToken cancellationToken = default(CancellationToken))
    {
        await Task.Delay(500, cancellationToken);//ネットワークトラフィックをシミュレートします。
        return new Tuple<int, IReadOnlyList<Customer>>(TotalCount, Enumerable.Range(startingIndex, count).Select(i => new Customer(i)).ToList());
    }
}

This is the code snippet for Android application:

Android
コードのコピー 
public class VirtualModeDataCollection : C1VirtualDataCollection<Customer>
{
    public int TotalCount { get; set; } = 1_000;

    protected override async Task<Tuple<int, IReadOnlyList<Customer>>> GetPageAsync(int pageIndex, int startingIndex, int count, IReadOnlyList<SortDescription> sortDescriptions = null, FilterExpression filterExpression = null, CancellationToken cancellationToken = default(CancellationToken))
    {
        await Task.Delay(500, cancellationToken);//ネットワークトラフィックをシミュレートします。
        return new Tuple<int, IReadOnlyList<Customer>>(TotalCount, Enumerable.Range(startingIndex, count).Select(i => new Customer(i)).ToList());
    }
}

This is the code snippet for iOS application:

iOS
コードのコピー 
public class VirtualModeDataCollection : C1VirtualDataCollection<Customer>
{
    public int TotalCount { get; set; } = 1_000;

    protected override async Task<Tuple<int, IReadOnlyList<Customer>>> GetPageAsync(int pageIndex, int startingIndex, int count, IReadOnlyList<SortDescription> sortDescriptions = null, FilterExpression filterExpression = null, CancellationToken cancellationToken = default(CancellationToken))
    {
        await Task.Delay(500, cancellationToken);//ネットワークトラフィックをシミュレートします。
        return new Tuple<int, IReadOnlyList<Customer>>(TotalCount, Enumerable.Range(startingIndex, count).Select(i => new Customer(i)).ToList());
    }
}

The GetPageAsync method of C1CursorDataCollection returns the items in the page as well as a token to the next page. This method uses startingIndex, pageToken, count, sortDescriptions and filterExpression as the parameters. The parameters startingIndex denotes the index where the returned items will be inserted, pageToken denotes the token of the requesting page and count denotes the desired number of items to be returned. To know more about the method and its parameters, refer this topic in detail. The GetPageAsync method returns pages with starting index (startingIndex) along with a list of the requested number of items (PageCount) starting from an index (startingIndex).

The following code snippet depicts the use of C1CursorDataCollection class:

This is the C# code snippet for WinForms application:

C#
コードのコピー 
public class MyCursorDataCollection : C1CursorDataCollection<Customer>
{
    protected override async Task<Tuple<string, IReadOnlyList<Customer>>> GetPageAsync(

        int startingIndex,

        string pageToken,

        int? count = null,

        IReadOnlyList<SortDescription> sortDescriptions = null,

        FilterExpression filterExpresssion = null,

        CancellationToken cancellationToken = default(CancellationToken));
    {
        var client = new HttpClient();

    var(items, token) = await client.GetAsync("http://www.contoso.com/myservice");

        return new Tuple<string, IReadOnlyList<Customer>>(token, items);
    }
}

This is the VB code snippet for WinForms application:

VB
コードのコピー 
Public Class MyCursorDataCollection
    Inherits C1CursorDataCollection(Of Customer)

    Protected Overrides Async Function GetPageAsync(ByVal startingIndex As Integer, ByVal pageToken As String, ByVal Optional count As Integer? = Nothing, ByVal Optional sortDescriptions As IReadOnlyList(Of SortDescription) = Nothing, ByVal Optional filterExpresssion As FilterExpression = Nothing, ByVal Optional cancellationToken As CancellationToken = Nothing) As Task(Of Tuple(Of String, IReadOnlyList(Of Customer)))


        Dim client = New HttpClient()
    End Function
    Private Sub New(ByVal _ As items, ByVal _ As token)
    End Sub

    Private Function GetAsync() As await
        Return New Tuple(Of String, IReadOnlyList(Of Customer))(token, items)
    End Function

End Class

This is the C# code snippet for WPF applications:

C#
コードのコピー 
public class CursorDataCollection : C1CursorDataCollection<Customer>
{      
    public int PageCount { get; set; } = 50;
    protected override async Task<Tuple<string, IReadOnlyList<Customer>>> GetPageAsync(int startingIndex, string pageToken, int? count = null, IReadOnlyList<SortDescription> sortDescriptions = null, FilterExpression filterExpression = null, CancellationToken cancellationToken = default)
    {
        await Task.Delay(500, cancellationToken);//ネットワークトラフィックをシミュレートします。
        return new Tuple<string, IReadOnlyList<Customer>>(startingIndex.ToString(), Enumerable.Range(startingIndex, PageCount).Select(i => new Customer(i)).ToList());
    }       
}

This is the VB code snippet for WPF applications:

VB
コードのコピー 
Public Class CursorDataCollection
    Inherits C1CursorDataCollection(Of Customer)
    Public Property PageCount As Integer = 50
    Protected Overrides Async Function GetPageAsync(startingIndex As Integer, pageToken As String, Optional count As Integer? = Nothing, Optional sortDescriptions As IReadOnlyList(Of SortDescription) = Nothing, Optional filterExpression As FilterExpression = Nothing, Optional cancellationToken As CancellationToken = Nothing) As Task(Of Tuple(Of String, IReadOnlyList(Of Customer)))
        Await Task.Delay(500, cancellationToken)
        Return New Tuple(Of String, IReadOnlyList(Of Customer))(startingIndex.ToString(), Enumerable.Range(startingIndex, PageCount).[Select](Function(i) New Customer(i)).ToList())
    End Function
End Class

This is the code snippet for Xamarin Forms:

Xamarin Forms
コードのコピー 
public class SimpleOnDemandDataCollection : C1CursorDataCollection<MyDataItem>
{
    public SimpleOnDemandDataCollection()
    {
        PageSize = 30;
    }

    public int PageSize { get; set; }

    protected override async Task<Tuple<string, IReadOnlyList<MyDataItem>>> GetPageAsync(int startingIndex, string pageToken, int? count = null, IReadOnlyList<SortDescription> sortDescriptions = null, FilterExpression filterExpresssion = null, CancellationToken cancellationToken = default(CancellationToken))
    {
        return await Task.Run(() =>
        {
            // 新しいアイテムのページを作成します。
            var newItems = new List<MyDataItem>();
            for (int i = 0; i < this.PageSize; i++)
            {
                newItems.Add(new MyDataItem(startingIndex + i));
            }

            return new Tuple<string, IReadOnlyList<MyDataItem>>("token not used", newItems);
        });
    }
}

This is the code snippet for Android applications:

Android
コードのコピー 
public class OnDemandDataCollection : C1CursorDataCollection<MyDataItem>
{
    private bool hasMoreItems = true;

    public override bool HasMoreItems
    { get { return hasMoreItems; } }

    const int MAX = 100;
    int current;
    public int PageSize { get; set; }
    public OnDemandDataCollection()
    {
        PageSize = 20;
    }
    protected override async Task<Tuple<string, IReadOnlyList<MyDataItem>>>
    GetPageAsync(int startingIndex, string pageToken, int? count = null,
    IReadOnlyList<SortDescription> sortDescriptions = null, FilterExpression
            filterExpression = null, CancellationToken cancellationToken = default(CancellationToken))
    {
        if (startingIndex + PageSize >= MAX)
        {
            hasMoreItems = false;
        }

        var newItems = new List<MyDataItem>();
        await Task.Run(() =>
        {
            for (int i = 0; i < this.PageSize; i++)
            {
                //Webサービスからデータを取得するなど、長い操作を模倣します。
                Thread.Sleep(100);

                newItems.Add(new MyDataItem(startingIndex + i));
                current++;
            }
        });
        return new Tuple<string, IReadOnlyList<MyDataItem>>("token not used", newItems);
    }

}

This is the code snippet for iOS applications:

iOS
コードのコピー 
public class OnDemandDataCollection : C1CursorDataCollection<MyDataItem>
{
    private bool hasMoreItems = true;

    public override bool HasMoreItems
    { get { return hasMoreItems; } }

    const int MAX = 100;
    int current;
    public int PageSize { get; set; }
    public OnDemandDataCollection()
    {
        PageSize = 20;
    }
    protected override async Task<Tuple<string, IReadOnlyList<MyDataItem>>> GetPageAsync(int startingIndex, string pageToken, int? count = null, IReadOnlyList<SortDescription> sortDescriptions = null, FilterExpression filterExpression = null, CancellationToken cancellationToken = default(CancellationToken))
    {
        //さらに読み込むアイテムはありません。
        if (startingIndex + PageSize >= MAX)
        {
            hasMoreItems = false;
        }

        var newItems = new List<MyDataItem>();
        await Task.Run(() =>
        {
            for (int i = 0; i < this.PageSize; i++)
            {
                //Webサービスからデータを取得するなど、長い時間操作を模倣します。
                Thread.Sleep(100);
                newItems.Add(new MyDataItem(startingIndex + i));
                current++;
            }
        });
        return new Tuple<string, IReadOnlyList<MyDataItem>>("token not used", newItems);
    }
}

Similarly, when the user clicks the Cursor button in the application, it gets a cursor-based data collection. The user can set a cursor-based data collection as the grid's ItemsSource, and call the LoadMoreItemsAsync method of C1CursorDataCollection to asynchronously load the records.

Private Sub btn_Virtualization_Click(sender As Object, e As RoutedEventArgs)
    cursorDataCollection = Nothing

    virtualDataCollection = New VirtualModeDataCollection() With {
        .Mode = VirtualDataCollectionMode.Manual
    }
    grid.ItemsSource = New C1CollectionView(virtualDataCollection)
    virtualDataCollection.LoadAsync(0, 0)
End Sub
Private Sub btn_Cursor_Click(sender As Object, e As RoutedEventArgs)
    virtualDataCollection = Nothing
    cursorDataCollection = New CursorDataCollection()
    grid.ItemsSource = New C1CollectionView(cursorDataCollection)
    cursorDataCollection.LoadMoreItemsAsync()
End Sub

private void btn_Virtualization_Click(object sender, RoutedEventArgs e)
{
    cursorDataCollection = null;
    virtualDataCollection = new VirtualModeDataCollection() { Mode = VirtualDataCollectionMode.Manual, PageSize = 50 };
    grid.ItemsSource = new C1CollectionView(virtualDataCollection);
    _ = virtualDataCollection.LoadAsync(0, 10);
}     
private void btn_Cursor_Click(object sender, RoutedEventArgs e)
{
    virtualDataCollection = null;
    cursorDataCollection = new CursorDataCollection();
    grid.ItemsSource = new C1CollectionView(cursorDataCollection);
    cursorDataCollection.LoadMoreItemsAsync();
}

In WPF application, the MS DataGrid control iterates the collection, which breaks the virtualization of C1VirtualDataCollection. In this case, the user has to handle the ScrollChanged event to load more pages when scrolling to the bottom. In the code below, the ScrollChanged event is called, such that if the grid is bound to a pagination-based data virtualization collection, the next record for the new viewport range are loaded asynchronously using the LoadAsync method of C1VirtualDataCollection. Likewise, if the grid is bound to a cursor-based data virtualization collection and the new viewport range is greater than the current scroll viewer area, then more records for the new viewport are loaded asynchronously using the LoadMoreItemsAsync method of C1CursorDataCollection.

Private Sub grid_ScrollChanged(sender As Object, e As ScrollChangedEventArgs)
    If virtualDataCollection IsNot Nothing Then
        virtualDataCollection.LoadAsync(CInt(e.VerticalOffset), CInt((e.VerticalOffset + e.ViewportHeight)))
    End If

    If cursorDataCollection IsNot Nothing AndAlso e.VerticalOffset + e.ViewportHeight >= e.ExtentHeight Then
        cursorDataCollection.LoadMoreItemsAsync(CInt(e.ViewportHeight))
    End If
End Sub

private void grid_ScrollChanged(object sender, ScrollChangedEventArgs e)
{
    if (virtualDataCollection != null)
    {
        _ = virtualDataCollection.LoadAsync((int)e.VerticalOffset, (int)(e.VerticalOffset + e.ViewportHeight));
    }

    if (cursorDataCollection != null && e.VerticalOffset + e.ViewportHeight >= e.ExtentHeight)
    {
        _ = cursorDataCollection.LoadMoreItemsAsync((int)e.ViewportHeight);
    }
}

 

 

Copyright © MESCIUS, Inc. All rights reserved.