Skip to content

Commit 7639558

Browse files
gh-160556: Fix quadratic time in csv.Sniffer with long runs of spaces
Sniffer._split_dormant() tried a match at every space of a run of spaces when the space was a candidate delimiter, and rescanned the rest of the run each time. A match starting at a space preceded by a space is never the leftmost one, so skip those positions with a lookbehind. The results do not change. The regression was introduced in gh-83273 (25fea6d).
1 parent dc28c3a commit 7639558

3 files changed

Lines changed: 14 additions & 2 deletions

File tree

‎Lib/csv.py‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -437,9 +437,12 @@ def _split_dormant(self, part, quotechar, combos):
437437
while remaining:
438438
# Include only the delimiters not found yet, so that the
439439
# search skips over the found ones; the compiled patterns
440-
# come from the re cache.
440+
# come from the re cache. The lookbehind skips matches
441+
# starting inside a run of spaces: an equivalent match
442+
# starts at the beginning of the run, and rescanning the
443+
# run from every space in it would take quadratic time.
441444
cls = re.escape(''.join(sorted(remaining)))
442-
m = re.compile(fr'(?:^|([\r\n{cls}]))'
445+
m = re.compile(fr'(?:^|([\r\n{cls}])(?<! ))'
443446
fr' *{quotechar}').search(part, pos)
444447
if m is None:
445448
break

‎Lib/test/test_csv.py‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1631,6 +1631,13 @@ def test_sniff_single_column_quoted(self):
16311631
self.assertRaisesRegex(csv.Error, "Could not determine delimiter",
16321632
sniffer.sniff, sample)
16331633

1634+
def test_sniff_long_run_of_spaces(self):
1635+
# gh-160556: a long run of spaces not followed by a quote character
1636+
# used to take quadratic time.
1637+
sniffer = csv.Sniffer()
1638+
sample = ('a,b' + ' ' * 100_000 + 'c"\n') * 4
1639+
self.assertEqual(sniffer.sniff(sample).delimiter, ',')
1640+
16341641
def test_sniff_non_ascii_delimiter(self):
16351642
# gh-111820: an explicitly requested delimiter can be non-ASCII.
16361643
sniffer = csv.Sniffer()
Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
Fix quadratic time in :meth:`csv.Sniffer.sniff` for samples with long runs
2+
of spaces. The regression was introduced in 3.16.0a1.

0 commit comments

Comments
 (0)