Introduction
The most efficient way to merge multiple lists and remove duplicates in Java is to add all elements to a LinkedHashSet (preserves insertion order) or a HashSet (unordered). For Java 8+, use Stream.concat() or Stream.of().flatMap() with .distinct(). The Set approach runs in O(n) time because HashSet.add() is O(1) amortized. The Stream.distinct() approach is equally efficient internally but more readable. For sorted output, use a TreeSet instead.
Using LinkedHashSet (Preserves Order)
1import java.util.*;
2
3List<String> list1 = Arrays.asList("apple", "banana", "cherry");
4List<String> list2 = Arrays.asList("banana", "date", "cherry");
5List<String> list3 = Arrays.asList("elderberry", "apple", "fig");
6
7// LinkedHashSet preserves insertion order and removes duplicates
8Set<String> merged = new LinkedHashSet<>();
9merged.addAll(list1);
10merged.addAll(list2);
11merged.addAll(list3);
12
13List<String> result = new ArrayList<>(merged);
14System.out.println(result);
15// [apple, banana, cherry, date, elderberry, fig]
LinkedHashSet maintains the order in which elements were first inserted. Duplicates are silently ignored. This is the most common approach when order matters.
Using HashSet (Fastest, Unordered)
1import java.util.*;
2
3List<Integer> list1 = Arrays.asList(1, 2, 3, 4);
4List<Integer> list2 = Arrays.asList(3, 4, 5, 6);
5List<Integer> list3 = Arrays.asList(5, 6, 7, 8);
6
7Set<Integer> merged = new HashSet<>();
8merged.addAll(list1);
9merged.addAll(list2);
10merged.addAll(list3);
11
12List<Integer> result = new ArrayList<>(merged);
13System.out.println(result);
14// [1, 2, 3, 4, 5, 6, 7, 8] — order not guaranteed
HashSet is the fastest option with O(1) average add/contains operations. Use it when the output order does not matter.
Using TreeSet (Sorted Output)
1import java.util.*;
2
3List<String> list1 = Arrays.asList("cherry", "apple");
4List<String> list2 = Arrays.asList("banana", "apple");
5
6Set<String> merged = new TreeSet<>();
7merged.addAll(list1);
8merged.addAll(list2);
9
10List<String> result = new ArrayList<>(merged);
11System.out.println(result);
12// [apple, banana, cherry] — sorted alphabetically
TreeSet keeps elements sorted using natural ordering or a custom Comparator. Each add is O(log n) instead of O(1), making it slower than HashSet for large lists.
Using Java 8 Streams
1import java.util.*;
2import java.util.stream.*;
3
4List<String> list1 = Arrays.asList("apple", "banana");
5List<String> list2 = Arrays.asList("banana", "cherry");
6List<String> list3 = Arrays.asList("cherry", "date");
7
8// Stream.of + flatMap + distinct
9List<String> result = Stream.of(list1, list2, list3)
10 .flatMap(Collection::stream)
11 .distinct()
12 .collect(Collectors.toList());
13
14System.out.println(result);
15// [apple, banana, cherry, date]
flatMap flattens the stream of lists into a single stream of elements. distinct() removes duplicates using equals() and hashCode(). This preserves encounter order for ordered streams.
Stream.concat for Two Lists
1import java.util.*;
2import java.util.stream.*;
3
4List<Integer> list1 = Arrays.asList(1, 2, 3);
5List<Integer> list2 = Arrays.asList(2, 3, 4);
6
7List<Integer> result = Stream.concat(list1.stream(), list2.stream())
8 .distinct()
9 .collect(Collectors.toList());
10
11System.out.println(result);
12// [1, 2, 3, 4]
Stream.concat merges exactly two streams. For more than two lists, use Stream.of(...).flatMap() or chain multiple concat calls.
Merging with Custom Objects
1import java.util.*;
2import java.util.stream.*;
3
4class Product {
5 String id;
6 String name;
7
8 Product(String id, String name) { this.id = id; this.name = name; }
9
10 @Override
11 public boolean equals(Object o) {
12 if (this == o) return true;
13 if (!(o instanceof Product)) return false;
14 return id.equals(((Product) o).id);
15 }
16
17 @Override
18 public int hashCode() { return id.hashCode(); }
19
20 @Override
21 public String toString() { return name; }
22}
23
24List<Product> store1 = Arrays.asList(
25 new Product("P1", "Laptop"), new Product("P2", "Phone"));
26List<Product> store2 = Arrays.asList(
27 new Product("P2", "Phone"), new Product("P3", "Tablet"));
28
29// Dedup by product ID using equals/hashCode
30List<Product> merged = Stream.of(store1, store2)
31 .flatMap(Collection::stream)
32 .distinct()
33 .collect(Collectors.toList());
34
35System.out.println(merged);
36// [Laptop, Phone, Tablet]
For custom objects, distinct() and Set use equals() and hashCode(). Both methods must be overridden consistently — objects that are equals() must have the same hashCode().
Dedup by Specific Field (Without equals Override)
1import java.util.*;
2import java.util.stream.*;
3import java.util.concurrent.ConcurrentHashMap;
4import java.util.function.Function;
5
6// Deduplicate by a specific field without overriding equals
7List<Product> merged = Stream.of(store1, store2)
8 .flatMap(Collection::stream)
9 .filter(distinctByKey(p -> p.id))
10 .collect(Collectors.toList());
11
12// Helper method
13static <T> java.util.function.Predicate<T> distinctByKey(
14 Function<? super T, ?> keyExtractor) {
15 Set<Object> seen = ConcurrentHashMap.newKeySet();
16 return t -> seen.add(keyExtractor.apply(t));
17}
This stateful filter tracks seen keys and removes duplicates based on any property, without modifying the class itself.
1// For N total elements across all lists:
2
3// HashSet approach — O(N) time, O(N) space
4Set<T> set = new HashSet<>();
5lists.forEach(set::addAll);
6
7// LinkedHashSet — O(N) time, O(N) space (slightly more overhead)
8Set<T> set = new LinkedHashSet<>();
9lists.forEach(set::addAll);
10
11// TreeSet — O(N log N) time, O(N) space
12Set<T> set = new TreeSet<>();
13lists.forEach(set::addAll);
14
15// Stream.distinct() — O(N) time, O(N) space (uses HashSet internally)
16Stream.of(lists).flatMap(Collection::stream).distinct();
17
18// Naive nested loop — O(N²) time, O(N) space — avoid
Common Pitfalls
Missing equals and hashCode overrides: HashSet, LinkedHashSet, and Stream.distinct() all rely on equals() and hashCode() for duplicate detection. Without overriding both methods in custom classes, objects with identical fields are treated as distinct because the default Object.equals() uses reference identity.
Using List.addAll without deduplication: list1.addAll(list2) merges lists but keeps all duplicates. This is a merge without dedup. Always funnel through a Set or use .distinct() to eliminate duplicates.
Modifying the source lists: Collections.addAll and List.addAll modify the target list in place. If list1 is unmodifiable (from Arrays.asList or List.of), this throws UnsupportedOperationException. Create a new ArrayList or use streams to avoid mutation.
Inconsistent equals and hashCode: If two objects are equals() but have different hashCode() values, HashSet may store both, failing to deduplicate. Always ensure that equal objects produce the same hash code.
Losing insertion order with HashSet: HashSet does not preserve insertion order. If the first occurrence order matters (e.g., priority from list1 over list2), use LinkedHashSet instead.
Summary
Use LinkedHashSet to merge lists with duplicate removal while preserving insertion order
Use HashSet when order does not matter — it is the fastest option
Use TreeSet when you need sorted output after merging
Use Stream.of(...).flatMap().distinct() for a functional, readable approach (Java 8+)
Override equals() and hashCode() in custom classes for proper duplicate detection
For field-based dedup without overriding equals, use a distinctByKey filter with a ConcurrentHashMap
