Getting All Child ListItem Objects in a Folder

[adamfisher]

I tried creating a method to get all of the child items (recursively) in a given folder in this previous discussion and came up with a CAML query to do it but it seems to have gotten slower over time.

So I came up with a new approach that seems to be way faster. Initial testing shows the extension method below takes about 50-70% less time to grab all ListItem objects and return themfor a given folder.

✅ I was then able to successfully update the metadata on all of the items in one go with no problem.
❌ My problem is I'm not able to load the file names and relative paths into each item that gets returned from this method.

I feel like I have a decent grasp of the differences between LoadAsync() loading in the SP context, GetAsync() being a direct call to get an item, and EnsurePropertiesAsync() which makes calls on the current item. I've tried variations of these but they do not seem to work when trying to load the Name and ServerRelativeUrl of the associated File or Folder object associated with the ListItem.

/// <summary>
/// Gets all the child items within the specified folder up to the maximum depth.
/// </summary>
/// <param name="folder">The folder to scan for items.</param>
/// <param name="maxDepth">The maximum depth to return items for.</param>
/// <param name="objectTypes">The type of items to return.</param>
public static async Task<List<IListItem>> GetChildItemsV2Async(this IFolder folder,
	int maxDepth = 20,
	params FileSystemObjectType[]? objectTypes)
{
	List<IListItem> items = new();
	Queue<(IFolder folder, int depth)> queue = new();
	objectTypes = objectTypes.IsNullOrEmpty() ? new[] { FileSystemObjectType.File, FileSystemObjectType.Folder } : objectTypes;
	queue.Enqueue((folder, 0));

	while (queue.Count > 0)
	{
		(IFolder currentFolder, int currentLevel) = queue.Dequeue();

		// Avoid going deeper than the maximum depth
		if (currentLevel > maxDepth) continue;

		// Load current folder's properties and files
		currentFolder = await currentFolder.PnPContext.Web.GetFolderByServerRelativeUrlAsync(
			currentFolder.ServerRelativeUrl,
			f => f.Name,
			f => f.ServerRelativeUrl,
			f => f.ListItemAllFields.QueryProperties(
				li => li.ContentType,
				li => li.File.QueryProperties(ff => ff.Name, ff => ff.ServerRelativeUrl),
				li => li.Folder.QueryProperties(ff => ff.Name, ff => ff.ServerRelativeUrl),
				li => li.FileSystemObjectType,
				li => li.FieldValuesForEdit),
			f => f.Folders.QueryProperties(
				p => p.Name,
				p => p.ServerRelativeUrl,
				p => p.ListItemAllFields.QueryProperties(
					li => li.ContentType,
					li => li.File.QueryProperties(ff => ff.Name, ff => ff.ServerRelativeUrl),
					li => li.Folder.QueryProperties(ff => ff.Name, ff => ff.ServerRelativeUrl),
					li => li.FileSystemObjectType,
					li => li.FieldValuesForEdit)),
			f => f.Files.QueryProperties(
				p => p.Name,
				p => p.ServerRelativeUrl,
				p => p.ListItemAllFields.QueryProperties(
					li => li.ContentType,
					li => li.File.QueryProperties(ff => ff.Name, ff => ff.ServerRelativeUrl),
					li => li.Folder.QueryProperties(ff => ff.Name, ff => ff.ServerRelativeUrl),
					li => li.FileSystemObjectType,
					li => li.FieldValuesForEdit)));

		// Add root folder passed in always
		if (items.Count == 0 && objectTypes!.Contains(FileSystemObjectType.Folder))
		{
			items.Add(folder.ListItemAllFields);
		}

		// Enqueue child folders for processing
		foreach (IFolder? childFolder in currentFolder.Folders.AsRequested())
		{
			queue.Enqueue((childFolder, currentLevel + 1));
		}

		// Process child folders
		if (objectTypes!.Contains(FileSystemObjectType.Folder))
		{
			List<IListItem> folderItems = currentFolder.Folders.AsRequested().Select(f => f.ListItemAllFields).ToList();
			items.AddRange(folderItems);
		}

		// Process child files
		if (objectTypes!.Contains(FileSystemObjectType.File))
		{
			List<IListItem> fileItems = currentFolder.Files.AsRequested().Select(f => f.ListItemAllFields).ToList();
			items.AddRange(fileItems);
		}
	}
	return items;
}

[adamfisher]

@jansenbe am I doing something wrong with the way I load the Name and ServerRelativeUrl for the file or folder? I want those brought back because I use them throughout the rest of the code elsewhere.

[adamfisher]

Ok after beating on this some more this seems to have worked:

/// <summary>
/// Gets all the child items within the specified folder up to the maximum depth.
/// </summary>
/// <param name="folder">The folder to scan for items.</param>
/// <param name="maxDepth">The maximum depth to return items for.</param>
/// <param name="objectTypes">The type of items to return.</param>
public static async Task<List<IListItem>> GetChildItemsAsync(this IFolder folder,
	int maxDepth = 20,
	params FileSystemObjectType[]? objectTypes)
{
	List<IListItem> items = new();
	Queue<(IFolder folder, int depth)> queue = new();
	objectTypes = objectTypes.IsNullOrEmpty() ? new[] { FileSystemObjectType.File, FileSystemObjectType.Folder } : objectTypes;
	queue.Enqueue((folder, 0));

	while (queue.Count > 0)
	{
		(IFolder currentFolder, int currentLevel) = queue.Dequeue();

		// Avoid going deeper than the maximum depth
		if (currentLevel > maxDepth) continue;

		// Load current folder's properties and files
		currentFolder = await currentFolder.PnPContext.Web.GetFolderByServerRelativeUrlAsync(
			currentFolder.ServerRelativeUrl,
			f => f.Name,
			f => f.ServerRelativeUrl,
			p => p.ListItemAllFields.QueryProperties(
				li => li.File.QueryProperties(ff => ff.Name, ff => ff.ServerRelativeUrl),
				li => li.FieldValuesForEdit),
			f => f.Folders.QueryProperties(
				p => p.Name,
				p => p.ServerRelativeUrl,
				p => p.ListItemAllFields.QueryProperties(
					li => li.File.QueryProperties(ff => ff.Name, ff => ff.ServerRelativeUrl),
					li => li.FieldValuesForEdit)
				),
			f => f.Files.QueryProperties(
				p => p.Name,
				p => p.ServerRelativeUrl,
				p => p.ListItemAllFields.QueryProperties(
					li => li.File.QueryProperties(ff => ff.Name, ff => ff.ServerRelativeUrl),
					li => li.FieldValuesForEdit)
				)
			);

		// Add root folder passed in always
		if (items.Count == 0 && objectTypes!.Contains(FileSystemObjectType.Folder))
		{
			items.Add(folder.ListItemAllFields);
		}

		// Enqueue child folders for processing
		foreach (IFolder? childFolder in currentFolder.Folders.AsRequested())
		{
			queue.Enqueue((childFolder, currentLevel + 1));
		}

		// Process child folders
		if (objectTypes!.Contains(FileSystemObjectType.Folder))
		{
			List<IListItem> folderItems = currentFolder.Folders.AsRequested().Select(f => f.ListItemAllFields).ToList();
			items.AddRange(folderItems);
		}

		// Process child files
		if (objectTypes!.Contains(FileSystemObjectType.File))
		{
			List<IListItem> fileItems = currentFolder.Files.AsRequested().Select(f => f.ListItemAllFields).ToList();
			items.AddRange(fileItems);
		}
	}

	return items;
}

[Amwins-Bass-Guy]

Do you know whether there is an upper limit to how many Files are returned for the Folder?

[adamfisher]

I'm not aware of any limits but I guess you could run into some if you have a folder with a ton of items in it. This method traverses and grabs everything at each folder level so not likely to run into a limit. I think I've seen elsewhere in the SDK where there is a default around 2,000 items but can't remember so maybe if you have a folder with that many files in it you may come across an issue and have to query it differently.