splitting a string based on multiple char delimiters
ML System Design practice on Codemia
Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.
Introduction
If you need to split a string on more than one delimiter, the right tool is usually a regular expression. The important distinction is whether your delimiters are single characters such as commas and semicolons, or multi-character tokens such as -- and ||, because the regex pattern should reflect that difference.
Use Regex Alternation for Multiple Delimiters
In Python, re.split is the standard answer.
This returns:
The pattern uses alternation, meaning “split on delimiter A or delimiter B.” This is the right model when the delimiters are multi-character strings.
Use a Character Class Only for Single-Character Delimiters
If your delimiters are single characters, a character class is shorter.
Here, the pattern means “one or more of comma, semicolon, or space.”
That is different from multi-character token splitting. A character class cannot represent whole delimiter strings such as -- as a single unit.
Keep Empty Fields in Mind
Splitting can produce empty strings when delimiters appear next to each other or at the ends.
Depending on the input, you may need to filter empty pieces.
Whether that is correct depends on whether empty fields are meaningful in your data format.
Escape Delimiters Carefully
Some delimiters contain regex metacharacters. For example, |, ., ?, +, and * all have special meaning in regex patterns.
That is why || in a delimiter pattern must be escaped as \|\| rather than written literally in the regex.
If the delimiters come from outside input, re.escape is often safer than writing the pattern manually.
Use Plain split Only When the Problem Is Simple
If you have only one delimiter, built-in split is simpler.
But once you need several delimiters, re.split is usually clearer than chaining multiple replacements or nested splits.
Common Pitfalls
- Using a character class when the delimiters are actually multi-character tokens.
- Forgetting to escape regex metacharacters such as
|or.. - Accidentally dropping meaningful empty fields after splitting.
- Writing a regex that matches too broadly and cuts the string in the wrong places.
- Using complex regex when a single plain
splitwould have solved the actual problem.
Summary
- Use
re.splitwhen you need to split on several delimiters. - Use alternation for multi-character delimiters such as
--|\|\|. - Use character classes only for sets of single-character delimiters.
- Decide explicitly whether empty pieces should be kept or filtered out.
- Escape delimiter text correctly when it contains regex-special characters.
Related reading
- Splitting a string into chunks of a certain size
- Splitting text into lines with maximum length
- String analysis
- string.ToLower and string.ToLowerInvariant
- Stripping out HTML tags from a string
- Stripping out HTML tags from a string
- Supervised Latent Dirichlet Allocation for Document Classification?
- Support vector machine or artificial neural network for text processing
.png&w=3840&q=75)
Tackling System Design Interview Problems
A short course that equips you with the skills to approach system design interviews methodically.
Start the free courseTrack what you have practised
A free account saves your progress, solutions and study plan across every problem on Codemia.
ML System Design practice on Codemia
Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.