$show=/label

Removing All Duplicate Values from ArrayList including Java 8 Streams

SHARE:

A Quick Guide to how to remove duplicate objects from ArrayList. Example program are demonstrated with Set, a new List with contains() method and Java 8 Streams. How to remove custom objects from ArrayList.

1. Overview

In this tutorial, We'll learn how to clean up the duplicate elements from ArrayList. Many of the scenarios we do encounter the same in real time projects. But, this can be resolved in many ways.

We will start with Set first then next using new List with using built in method contains() before adding each value to newList.

Next, will see duplicate elements removal with new java 8 concept Streams.

Then later, interestingly we have a list with Employee objects. Employee has id and name. Some Employee objects are added to list. Later realized that list has now duplicate employee based on the id. Now how to remove custom duplicate objects from ArrayList.

Removing All Duplicate Values from ArrayList including Java 8 Streams



we'll see how to clean up duplicate custom objects at end of this course.

In this post, We will execute all our code on below input list.

List fewMonths = new ArrayList<>();
fewMonths.add("JAN");
fewMonths.add("FEB");
fewMonths.add("MAR");
fewMonths.add("APR");
fewMonths.add("FEB");

Output:

Printing the duplicate list.

Duplicate List : [JAN, FEB, MAR, APR, FEB]

After removing duplicates List should be as below.

After removing duplicates : [JAN, FEB, MAR, APR]

2. Using Set

Set is a interface in Java which does not allow duplicates. One of Set implementations are Hashset and LinkedHashSet.

Hashset doesn't preserve the insertion order where as LinkedHashSet holds in the insertion order.

We need to just pass the duplicate fewMonths List to set while creating it's instance.

Set afterRemovingDuplicates = new LinkedHashSet<>(fewMonths);

Output:

After removing duplicates : [JAN, FEB, MAR, APR]

Order is preserved as in the input List.

Full Example Code

3. For Loop - with new List


In this approach,

A) Create a new List
B) Iterate the duplicate List
C) Take each value from duplicate list then checks in newList using contains() method whether this element is present in the newList.
D) If not present, add it to the newList
E) If present, do not add it to the newList.
F) Finally, newList is free from duplicates.

List newList = new ArrayList<>();

for (String month : fewMonths) {

 if (!newList.contains(month)) {
  newList.add(month);
 }
}


Full Example Code

4. Java 8 - Stream.distinct()

In this approach, We'll be using Java 8 Lambda Expressions to solve using Stream API. Stream API has a method distinct() which returns a stream of distinct elements by calling equals method.

List newMonthsListWithoutDuplicates = fewMonths.stream()
                .distinct()
                .collect(Collectors.toList());  

Full Example Code

5. Custom Objects

Let us take a look if List is having user defined objects rather than String or Wrapper objects in it.

For Example

Employee.java

class Employee {
 private int id;
 private String name;
 
 public Employee(int id, String name) {
  this.id = id;
  this.name = name;
 }
 
 // setters and getters
 
 @Override
 public String toString() {
  return "Employee [id=" + id + ", name=" + name + "]";
 }
 
}

Removing Duplicates using java 8:

Now adding 5 employee objects with id 100. We need to clean up the ArrayList and finally it should have only one Employee object which is unique. Below program is implemented using distinct() method.

public class RemovalUsingDistinct {

 public static void main(String[] args) {

  List fewMonths = new ArrayList<>();
  fewMonths.add(new Employee(100, "Jhon"));
  fewMonths.add(new Employee(100, "Jhon"));
  fewMonths.add(new Employee(100, "Jhon"));
  fewMonths.add(new Employee(100, "Jhon"));
  fewMonths.add(new Employee(100, "Jhon"));

  System.out.println("Duplicate List size: " + fewMonths.size());

  List newMonthsListWithoutDuplicates = fewMonths.stream().distinct().collect(Collectors.toList());
  System.out.println("After removing duplicates list size : "+newMonthsListWithoutDuplicates.size());
 }

}

Output:

Duplicate List size: 5
After removing duplicates list size : 5

Full Example Code

Observe the output and lists sizes are same before and after removal.

All above shown solutons will give same size as input.

In these types of scenarioes, We should know API internal working mechanism to solve.

Solution is just need to override the equals() and hashcode() method in Employee class as below.

@Override
public int hashCode() {
 // TODO Auto-generated method stub
 return this.id;
}

@Override
public boolean equals(Object obj) {
 Employee other = (Employee) obj;
 if (id != other.id)
  return false;
 return true;
}
 
Output:

now see the output

Duplicate List size: 5
After removing duplicates list size : 1

Full Example Code

6. Conclusion

In this tutorial, We've seen how easy to clean up duplicates from ArrayList using LinkedHashSet, new list using contains() mehtod and java 8 stream api distinct() method.

And more over demonstrated removing user defined duplicate objects from List.
All code snippet shown here are available on GitHub.

COMMENTS

BLOGGER

About Us

Author: Venkatesh - I love to learn and share the technical stuff.
Name

accumulo,1,ActiveMQ,2,Adsense,1,API,37,ArrayList,16,Arrays,7,Bean Creation,3,Bean Scopes,1,BiConsumer,1,Blogger Tips,1,Books,1,C Programming,1,Collection,5,Collections,22,Collector,1,Command Line,1,Compile Errors,1,Configurations,7,Constants,1,Control Statements,8,Conversions,6,Core Java,81,Corona India,1,Create,2,CSS,1,Date,3,Date Time API,4,Dictionary,1,Difference,1,Download,1,Eclipse,2,Efficiently,1,Error,1,Errors,1,Exception,1,Exceptions,3,Fast,1,Files,9,Float,1,Font,1,Form,1,Freshers,1,Function,3,Functional Interface,2,Garbage Collector,1,Generics,4,Git,4,Grant,1,Grep,1,HashMap,1,HomeBrew,2,HTML,2,HttpClient,2,Immutable,1,Installation,1,Interview Questions,5,Iterate,2,Jackson API,3,Java,29,Java 10,1,Java 11,5,Java 12,5,Java 13,2,Java 14,2,Java 8,66,Java 8 Difference,2,Java 8 Stream Conversions,2,java 8 Stream Examples,3,Java 9,1,Java Conversions,11,Java Design Patterns,1,Java Files,1,Java Program,2,Java Programs,65,java.lang,5,java.util. function,1,jQuery,1,Kotlin,10,Kotlin Conversions,3,Kotlin Programs,6,Lambda,1,lang,29,Leap Year,1,live updates,1,Logging,1,Mac OS,2,Math,1,Maven,1,Method References,1,Mockito,1,MongoDB,3,New Features,1,Operations,1,Optional,6,Oracle,5,Oracle 18C,1,Partition,1,Patterns,1,Programs,1,Property,1,Python,2,Quarkus,1,Read,1,Real Time,1,Recursion,2,Remove,2,Rest API,1,Schedules,1,Serialization,1,Servlet,1,Sort,1,Sorting Techniques,8,Spring,2,Spring Boot,23,Spring Email,1,Spring MVC,1,Streams,21,String,58,String Programs,9,String Revese,1,Swing,1,System,1,Tags,1,Threads,10,Tomcat,1,Tomcat 8,1,Troubleshoot,16,Unix,2,Updates,3,util,5,While Loop,1,
ltr
item
JavaProgramTo.com: Removing All Duplicate Values from ArrayList including Java 8 Streams
Removing All Duplicate Values from ArrayList including Java 8 Streams
A Quick Guide to how to remove duplicate objects from ArrayList. Example program are demonstrated with Set, a new List with contains() method and Java 8 Streams. How to remove custom objects from ArrayList.
https://1.bp.blogspot.com/-Qc1YU0dgdUg/XOutUfciXOI/AAAAAAAABjM/NKa4Uc9k4VYSLORmbjeKseE6a6zJBmitACLcBGAs/s320/Removing%2BAll%2BDuplicate%2BValues%2Bfrom%2BArrayList%2Bincluding%2BJava%2B8%2BStreams.PNG
https://1.bp.blogspot.com/-Qc1YU0dgdUg/XOutUfciXOI/AAAAAAAABjM/NKa4Uc9k4VYSLORmbjeKseE6a6zJBmitACLcBGAs/s72-c/Removing%2BAll%2BDuplicate%2BValues%2Bfrom%2BArrayList%2Bincluding%2BJava%2B8%2BStreams.PNG
JavaProgramTo.com
https://www.javaprogramto.com/2019/05/remove-duplicates-from-arraylist.html
https://www.javaprogramto.com/
https://www.javaprogramto.com/
https://www.javaprogramto.com/2019/05/remove-duplicates-from-arraylist.html
true
3124782013468838591
UTF-8
Loaded All Posts Not found any posts VIEW ALL Readmore Reply Cancel reply Delete By Home PAGES POSTS View All RECOMMENDED FOR YOU LABEL ARCHIVE SEARCH ALL POSTS Not found any post match with your request Back Home Sunday Monday Tuesday Wednesday Thursday Friday Saturday Sun Mon Tue Wed Thu Fri Sat January February March April May June July August September October November December Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec just now 1 minute ago $$1$$ minutes ago 1 hour ago $$1$$ hours ago Yesterday $$1$$ days ago $$1$$ weeks ago more than 5 weeks ago Followers Follow THIS PREMIUM CONTENT IS LOCKED STEP 1: Share to a social network STEP 2: Click the link on your social network Copy All Code Select All Code All codes were copied to your clipboard Can not copy the codes / texts, please press [CTRL]+[C] (or CMD+C with Mac) to copy Table of Content