How to check if items of a list exist in another list in c#?

  • (3 Pages)
  • +
  • 1
  • 2
  • 3

39 Replies - 1419 Views - Last Post: 16 October 2017 - 10:09 AM Rate Topic: -----

#31 JapanDave  Icon User is offline

  • D.I.C Regular

Reputation: 29
  • View blog
  • Posts: 366
  • Joined: 01-February 16

Re: How to check if items of a list exist in another list in c#?

Posted 14 October 2017 - 06:51 PM

OK, what I did was to add each matching set of files into a jagged array to group them in their separate parent directories, so as you can know which files came from where.

Then just loop through the jagged array one element at a time and do what you need to them,
This is the code I used, I tried to structure the LINQ so it was a little more easy to understand. You could go with nested for loops, but that gets confusing pretty quickly.

   //get all directories from basepath 
          string[] filesindirectory = Directory.GetDirectories(basePath);


         //Loop through each parent directory and get each matching xml file from it
            List<string[]> newList = filesindirectory.Select(folder => (from item in Directory.GetDirectories(folder, "meta", SearchOption.AllDirectories)
                            .Select(item => Directory.GetFiles(item, "*.xml"))
                            .ToList()
                            .SelectMany(x => x)
                        let sx = Directory.GetDirectories(folder, "xml", SearchOption.AllDirectories)
                            .Select(items => Directory.GetFiles(items, "*.xml"))
                            .ToList()
                            .SelectMany(s => s)
                            .Any(s => Path.GetFileName(s) == Path.GetFileName(item))
                        where sx
                        select item).ToArray()
                    .Concat((from xmlItem in Directory.GetDirectories(folder, "xml", SearchOption.AllDirectories)
                            .Select(item => Directory.GetFiles(item, "*.xml"))
                            .ToList()
                            .SelectMany(xs => xs)
                        let sx = Directory.GetDirectories(folder, "meta", SearchOption.AllDirectories)
                            .Select(items => Directory.GetFiles(items, "*.xml"))
                            .ToList()
                            .SelectMany(sc => sc)
                            .Any(sc => Path.GetFileName(sc) == Path.GetFileName(xmlItem))
                        where sx
                        select xmlItem).ToArray()))
                .Select(xmlFiles => xmlFiles.ToArray()).ToList();


            //loop through each element of the jagged array
            foreach (string[] path in newList)
            {
                for (int j = 0; j < path.Length / 2; j++)
                {
                    XDocument doc = Xdocument.Load(path[j]);
                    string name = doc.Root.Element("Emp").Element("lbl").Value;
                    XDocument doc2 = Xdocument.Load(path[(path.Length / 2) + j]);
                    doc2.Root.Element("Employee").SetElementValue("label", name);
                    doc2.Save(path[(path.Length / 2) + j]);
                }
            }


Edited: to make LINQ a little more readable and also to add missing variable.

This post has been edited by JapanDave: 14 October 2017 - 07:10 PM

Was This Post Helpful? 1
  • +
  • -

#32 Whateva_  Icon User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 42
  • Joined: 28-August 16

Re: How to check if items of a list exist in another list in c#?

Posted 14 October 2017 - 08:57 PM

Thank again JapanDave :rockon:/>
BTW, I'm lost at newList, it is just too much for a newbie I guess, but your code works like a charm :bigsmile:/>
Also, in your previous approach I added a try, catch to stop the program from crashing and just create a log file in the basePath with the name of the file and then continue to the next file, like below
string mylog=Regex.Replace(item, @"^.*\\", "");
				try
				{
					var metaDirectoryFile = metaDupFiles.FirstOrDefault(s => Path.GetFileName(s) == Path.GetFileName(item));
					if (string.IsNullOrEmpty(metaDirectoryFile)) return;
                    XDocument doc = Xdocument.Load(metaDirectoryFile);
                    var v = (from emp in doc.Descendants("lbl")
                             select emp).First().Value;
                    XDocument doc2 = Xdocument.Load(item);
                    doc2.Descendants("label").First().Value = v;
                    doc2.Save(item);
                   
                }
				catch (Exception ex)
				{
					string filePath = basePath + @"\Error.log";

					using (StreamWriter writer = new StreamWriter(filePath, true))
					{
						writer.WriteLine(mylog+":\t"+ex.Message);

					}
				}

I can't figure out how to add this try, catch in your new code though.

This post has been edited by Whateva_: 14 October 2017 - 08:58 PM

Was This Post Helpful? 0
  • +
  • -

#33 Whateva_  Icon User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 42
  • Joined: 28-August 16

Re: How to check if items of a list exist in another list in c#?

Posted 14 October 2017 - 09:02 PM

View PostSkydiver, on 14 October 2017 - 11:44 AM, said:

As for the Descendants issue within the XML itself, I suggest using an XPath query and use SelectSingleNode() or SelectNodes() to simplify targeting which element(s) you are interested in.

Does SelectSingleNode() or SelectNodes() require you to give the entire node tree or can I just give the name of the node, say SelectSingleNode("label") or SelectNodes("label") and it will get me the contents of the node regardless whether it a parent node of the file or child node or child node of a child node and so on... :dontgetit:
Was This Post Helpful? 0
  • +
  • -

#34 JapanDave  Icon User is offline

  • D.I.C Regular

Reputation: 29
  • View blog
  • Posts: 366
  • Joined: 01-February 16

Re: How to check if items of a list exist in another list in c#?

Posted 14 October 2017 - 09:15 PM

I am gathering that the item variable is from the foreach loop,

So the code should look like this.

      foreach (string[] path in newList)
            {
                for (int j = 0; j < path.Length / 2; j++)
                {
                    string mylog = Regex.Replace(path[(path.Length / 2) + j], @"^.*\\", "");
                    try
                    {
                        XDocument doc = Xdocument.Load(path[j]);
                        string name = doc.Root.Element("Emp").Element("lbl").Value;
                        XDocument doc2 = Xdocument.Load(path[(path.Length / 2) + j]);
                        doc2.Root.Element("Employee").SetElementValue("label", name);
                        doc2.Save(path[(path.Length / 2) + j]);
                    }
                    catch (Exception ex)
                    {
                        string filePath = basePath + @"\Error.log";

                        using (StreamWriter writer = new StreamWriter(filePath, true))
                        {
                            writer.WriteLine(mylog + ":\t" + ex.Message);

                        }
                    }
                }
            }


I probably should break down the code into smaller pieces, but that will make a huge chunk of code. If I get time and you are interested, I will post if you like?

And btw, you dn't need ReGex to get file name and extension, use this.

string mylog = Path.GetFileName(path[(path.Length / 2) + j]);

This post has been edited by JapanDave: 14 October 2017 - 09:22 PM

Was This Post Helpful? 0
  • +
  • -

#35 Whateva_  Icon User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 42
  • Joined: 28-August 16

Re: How to check if items of a list exist in another list in c#?

Posted 14 October 2017 - 09:25 PM

View PostJapanDave, on 14 October 2017 - 09:15 PM, said:

I probably should break down the code into smaller pieces, but that will make a huge chunk of code. If I get time and you are interested, I will post if you like?

I would really appreciate that.. :bigsmile:
Was This Post Helpful? 0
  • +
  • -

#36 JapanDave  Icon User is offline

  • D.I.C Regular

Reputation: 29
  • View blog
  • Posts: 366
  • Joined: 01-February 16

Re: How to check if items of a list exist in another list in c#?

Posted 14 October 2017 - 10:02 PM

OK, this is the code without LINQ,

You just can't get away from all the for loops doing it this way.

   string[] filesindirectory = Directory.GetDirectories(basePath);

            List<string[]> newList = new List<string[]>();

            foreach (var folder in filesindirectory)
            {
                List<string> xmlFolders = new List<string>();
                List<string> metaFolders = new List<string>();
                List<string> metaDupFiles = new List<string>();
                List<string> xmlDupFiles = new List<string>();

                //Get all paths in folders named xml
                foreach (var path in Directory.GetDirectories(folder, "xml", SearchOption.AllDirectories))
                {
                    foreach (var file in Directory.GetFiles(path, "*.xml"))
                    {
                        xmlFolders.Add(file);
                    }
                }


                //Get all paths in folder named meta
                foreach (var path in Directory.GetDirectories(folder, "meta", SearchOption.AllDirectories))
                {
                    foreach (var file in Directory.GetFiles(path, "*.xml"))
                    {
                        metaFolders.Add(file);
                    }
                }

                //Get dup paths for files in xml directory
                foreach (var item in metaFolders)
                {
                    var dup = xmlFolders.FirstOrDefault(s => Path.GetFileName(s) == Path.GetFileName(item));
                    if (dup != null && dup.Any()) 
                    {
                        xmlDupFiles.Add(dup);
                    }
                }

                //Get dup paths for files in meta directory
                foreach (var item in xmlFolders)
                {
                    var dup = metaFolders.FirstOrDefault(s => Path.GetFileName(s) == Path.GetFileName(item));
                    if (dup != null && dup.Any())
                    {
                        metaDupFiles.Add(dup);
                    }
                }
                
                var combinArray = metaDupFiles.Concat(xmlDupFiles).ToArray();

                newList.Add(combinArray);
            }


On a side not though, this may be faster than the LINQ way, but you would need 1000's of directories to start to see the difference.

This post has been edited by JapanDave: 14 October 2017 - 10:03 PM

Was This Post Helpful? 0
  • +
  • -

#37 Whateva_  Icon User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 42
  • Joined: 28-August 16

Re: How to check if items of a list exist in another list in c#?

Posted 14 October 2017 - 10:43 PM

Hi JapanDave,
Could you explain the your previous LINQ code (if you have time of course)

List<string[]> newList = filesindirectory.Select(folder => (from item in Directory.GetDirectories(folder, "meta", SearchOption.AllDirectories)
                         .Select(item => Directory.GetFiles(item, "*.xml"))
                         .ToList()
                         .SelectMany(x => x)
                     let sx = Directory.GetDirectories(folder, "xml", SearchOption.AllDirectories)
                         .Select(items => Directory.GetFiles(items, "*.xml"))
                         .ToList()
                         .SelectMany(s => s)
                         .Any(s => Path.GetFileName(s) == Path.GetFileName(item))
                     where sx
                     select item).ToArray()
                 .Concat((from xmlItem in Directory.GetDirectories(folder, "xml", SearchOption.AllDirectories)
                         .Select(item => Directory.GetFiles(item, "*.xml"))
                         .ToList()
                         .SelectMany(xs => xs)
                     let sx = Directory.GetDirectories(folder, "meta", SearchOption.AllDirectories)
                         .Select(items => Directory.GetFiles(items, "*.xml"))
                         .ToList()
                         .SelectMany(sc => sc)
                         .Any(sc => Path.GetFileName(sc) == Path.GetFileName(xmlItem))
                     where sx
                     select xmlItem).ToArray()))
             .Select(xmlFiles => xmlFiles.ToArray()).ToList();

I'm familiar with the LINQ operations from, in, select, where but not let, SelectMany, Any, Concat.

This post has been edited by Whateva_: 14 October 2017 - 10:45 PM

Was This Post Helpful? 0
  • +
  • -

#38 andrewsw  Icon User is online

  • the case is sol-ved
  • member icon

Reputation: 6374
  • View blog
  • Posts: 25,755
  • Joined: 12-December 12

Re: How to check if items of a list exist in another list in c#?

Posted 14 October 2017 - 11:21 PM

LINQ By Example
Was This Post Helpful? 1
  • +
  • -

#39 JapanDave  Icon User is offline

  • D.I.C Regular

Reputation: 29
  • View blog
  • Posts: 366
  • Joined: 01-February 16

Re: How to check if items of a list exist in another list in c#?

Posted 14 October 2017 - 11:23 PM

@Andrewsw, awesome tutorial!!! Never saw that before. +1 for link (pun not intended)

@Whateva, give me a bit and I will see how I can best explain it. In the mean time take a look at Andrewsw's tutorial.

This post has been edited by JapanDave: 14 October 2017 - 11:26 PM

Was This Post Helpful? 0
  • +
  • -

#40 Skydiver  Icon User is offline

  • Code herder
  • member icon

Reputation: 5884
  • View blog
  • Posts: 20,091
  • Joined: 05-May 12

Re: How to check if items of a list exist in another list in c#?

Posted 16 October 2017 - 10:09 AM

I know that I kept on hinting how much simpler this becomes because of the fixed structure of the directory names and filenames. Let me demonstrate the pseudo code involved in making it work:
struct XmlFilePair
{
    public string Data;
    public string Meta;

    public XmlFilePair(string data, string meta)
    {
        Data = data;
        Meta = meta;
    }
}

IEnumerable <XmlFilePair> GetXmlFilePairs(string basePath)
{
    var baseDirInfo = new DirectoryInfo(basePath);
    foreach(var dirInfo in baseDirInfo.GetDirectories("???-???-???"))
    {
        string parts = dirInfo.Split("-");
        if (!(IsAllDigits(parts[0]) && IsAllDigits(parts[1]) and && IsAllDigits(parts[2])))
            continue;

        var xmlFile = $"{parts[0]}.xml";
        var metaFile = Path.Combine(dirInfo.FullName, $@"meta\{xmlFile}");
        var dataFile = Path.Combine(dirInfo.FullName, $@"{parts[0]}\{parts[1]}\{parts[2]}\xml\{xmlFile}");

        if (File.Exists(metaFile) && File.Exists(dataFile))
            yield return new XmlFilePair(data : dataFile, meta : metaFile)
    }
}

bool IsAllDigits(string s)
{
    return s.All(ch => Char.IsDigit(ch));
}

void ProcessXmlFiles(string basePath)
{
     foreach(var pair in GetXmlFilePairs(basePath))
     {
         var meta = new XDocument(pair.Meta);
         var name = meta.XPathSelectElement("/Emp/lbl").Value;
         var data = new XDocument(pair.Data);
         data.XPathSelectElement("/Employee/label").Value = name;
         data.Save(pair.Data);
     }
}


Was This Post Helpful? 0
  • +
  • -

  • (3 Pages)
  • +
  • 1
  • 2
  • 3