Visual Basic File Duplicate Line Remover

Submitted by Yorkiebar on Wednesday, September 4, 2013 - 05:22.

Introduction: Hello, In this tutorial we will be making a simple duplicate line remover. This is made for text file lines but can be easily modified for many other purposes such as .csv duplicate value removers. Steps of Creation: Step 1: First we will create a new form and add: 1 - Button - To begin the duplicate line remover. 1 - Checkbox - To check if the program should overwrite the opened file Step 2: Now lets add a script to create OpenFileDialog to select the file to remove duplicate lines from:

    Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
        Dim fo As New OpenFileDialog
        fo.RestoreDirectory = True
        fo.Multiselect = False
        fo.Filter = "txt files (*.txt)|*.txt"
        fo.FilterIndex = 1
        fo.ShowDialog()
        If (Not fo.FileName = Nothing) Then
        End If
    End Sub

Step 3: Next let's create a new String List and add every line from the file to it. As we add them we will compare the line to the already added lines and if the line is not already added we will add it!

    Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
        Dim fo As New OpenFileDialog
        fo.RestoreDirectory = True
        fo.Multiselect = False
        fo.Filter = "txt files (*.txt)|*.txt"
        fo.FilterIndex = 1
        fo.ShowDialog()
        If (Not fo.FileName = Nothing) Then
            Dim lines As New List(Of String)
            Using sr As New System.IO.StreamReader(fo.FileName)
                While sr.Peek <> -1
                    Dim line As String = sr.ReadLine()
                    Dim isNew As Boolean = True
                    For Each found As String In lines
                        If (found = line) Then isNew = False
                    Next
                    If (isNew) Then lines.Add(sr.ReadLine())
                End While
            End Using
        End If
    End Sub

Step 3: Finally, let's create a SaveFileDialog if the Checkbox to overwite the file is not checked, otherwise we will write the values in the String List "lines" to the file we opened:

    Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
        Dim fo As New OpenFileDialog
        fo.RestoreDirectory = True
        fo.Multiselect = False
        fo.Filter = "txt files (*.txt)|*.txt"
        fo.FilterIndex = 1
        fo.ShowDialog()
        If (Not fo.FileName = Nothing) Then
            Dim lines As New List(Of String)
            Using sr As New System.IO.StreamReader(fo.FileName)
                While sr.Peek <> -1
                    Dim line As String = sr.ReadLine()
                    Dim isNew As Boolean = True
                    For Each found As String In lines
                        If (found = line) Then isNew = False
                    Next
                    If (isNew) Then lines.Add(sr.ReadLine())
                End While
            End Using
            Dim savePath As String
            If (CheckBox1.Checked) Then
                If (My.Computer.FileSystem.FileExists(fo.FileName)) Then
                    My.Computer.FileSystem.DeleteFile(fo.FileName)
                End If
                savePath = fo.FileName
            Else
                Dim fs As New SaveFileDialog
                fs.RestoreDirectory = True
                fs.Filter = "txt files (*.txt)|*.txt"
                fs.FilterIndex = 1
                fs.ShowDialog()
                If (fs.FileName = Nothing) Then
                    savePath = fo.FileName
                Else
                    savePath = fs.FileName
                End If
            End If
            Using sw As New System.IO.StreamWriter(savePath)
                For Each line As String In lines
                    sw.WriteLine(line)
                Next
            End Using
        End If
    End Sub

Project Completed! That's it! Below is the source code and you can download the project from the attached files below.

Public Class Form1
    Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
        Dim fo As New OpenFileDialog
        fo.RestoreDirectory = True
        fo.Multiselect = False
        fo.Filter = "txt files (*.txt)|*.txt"
        fo.FilterIndex = 1
        fo.ShowDialog()
        If (Not fo.FileName = Nothing) Then
            Dim lines As New List(Of String)
            Using sr As New System.IO.StreamReader(fo.FileName)
                While sr.Peek <> -1
                    Dim line As String = sr.ReadLine()
                    Dim isNew As Boolean = True
                    For Each found As String In lines
                        If (found = line) Then isNew = False
                    Next
                    If (isNew) Then lines.Add(sr.ReadLine())
                End While
            End Using
            Dim savePath As String
            If (CheckBox1.Checked) Then
                If (My.Computer.FileSystem.FileExists(fo.FileName)) Then
                    My.Computer.FileSystem.DeleteFile(fo.FileName)
                End If
                savePath = fo.FileName
            Else
                Dim fs As New SaveFileDialog
                fs.RestoreDirectory = True
                fs.Filter = "txt files (*.txt)|*.txt"
                fs.FilterIndex = 1
                fs.ShowDialog()
                If (fs.FileName = Nothing) Then
                    savePath = fo.FileName
                Else
                    savePath = fs.FileName
                End If
            End If
            Using sw As New System.IO.StreamWriter(savePath)
                For Each line As String In lines
                    sw.WriteLine(line)
                Next
            End Using
        End If
    End Sub
End Class

Add new comment

129 views

Visual Basic File Duplicate Line Remover

Tags

Add new comment