Answers

• 

  Anurag Srivastava

  Oct 11, 2020, 8:38:28 AM | Share

  • Facebook
  • Twitter
  • WhatsApp
  • Linkedin
  • Copy Link

  We can read the CSV file data and can push it to Elasticsearch using Logstash and for that, we need to create a Logstash configuration file. You need to write the following expression in the Logstash configuration file (for example the configuration file name: "crimes.conf"):

  input {
      file {
          path => "/home/user/Downloads/crimes_2001.csv"
          start_position => beginning
      }
  }
  filter {
      csv {
          columns => [
                  "ID",
                  "Case Number",
                  "Date",
                  "Block",
                  "IUCR",
                  "Primary Type",
                  "Description",
                  "Location Description",
                  "Arrest",
                  "Domestic",
                  "Beat",
                  "District",
                  "Ward",
                  "Community Area",
                  "FBI Code",
                  "X Coordinate",
                  "Y Coordinate",
                  "Year",
                  "Updated On",
                  "Latitude",
                  "Longitude",
                  "Location"
          ]
          separator => ","
          }
  }
  output {
      stdout
      {
          codec => rubydebug
      }
       elasticsearch {
          action => "index"
          hosts => ["127.0.0.1:9200"]
          index => "crimes"
      }
  }
  

  
  After creating the Logstash configuration file execute the configuration using the following command:

  /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/crimes.conf 
  

  
  

  This command will create the pipeline to read the CSV data and to write it into Elasticsearch. You can verify the index "crimes" creation in Elasticsearch by listing indices. 

  
  

  For more details, you can refer to my blog: "Loading CSV Data into Elasticsearch"

  
  

  In case of any queries please leave your comments.
  

  0 | 0