Dataset Equivalence Check

You are given two datasets, each represented as a list of files. Each file is a list of rows, and each row is a list of integer columns.

Determine whether the two datasets are equivalent.

Two datasets D1 and D2 are equivalent if and only if:

• Every row in D1 (across any file) appears in D2 (across any file), AND
• Every row in D2 appears in D1, AND
• Multiplicities match (a row appearing 3 times in D1 must appear 3 times in D2).

Two rows are equal only if they have the same columns in the same order (so [1,2,3] and [1,3,2] are NOT equal).

Examples:

` D1 = [[[1,2,3], [4,5,6]]] D2 = [[[4,5,6], [1,2,3]]] result = True (file boundaries don't matter, row order within a file doesn't matter)

D1 = [[[1,2,3]]] D2 = [[[1,3,2]]] result = False (column order is significant) `

Follow-up Questions:

• What if some "rare mismatches" should be tolerated (allow up to K rows of difference)?
• How would you stream this comparison if the datasets don't fit in memory?