Python script for ElasticSearch and Kibana APIs
Budget: $10 – $30 USD
I want to programmatically (w/ Python 3.10.5) query an ElasticSearch (v7.16.2) instances and:
1) Get a list of indices.
2) Get a list of index patterns
3) Get a list of fields identical to what Kibana would show under "Available Fields" for the index.
#3 has been the biggest frustration. I tried the "Field usage stats API" but there were always fields missing. Using the "Get field mapping API" returns everything in the mapping which is way too broad. How does Kibana know which fields are available for each index (the one shown under "available fields"?
These are the lines of code I used to pull the fields from each index.
field_usage_stats = es.indices.field_usage_stats(index=index) #too few fields returned
field_mapping = es.indices.get_mapping(index=index) #too many fields returned
Reference
https://elasticsearch-py.readthedocs.io/en/7.x/api.html
https://www.elastic.co/guide/en/elasticsearch/reference/7.16/field-usage-stats.html
https://www.elastic.co/guide/en/elasticsearch/reference/7.16/indices-get-mapping.html
1) Get a list of indices.
2) Get a list of index patterns
3) Get a list of fields identical to what Kibana would show under "Available Fields" for the index.
#3 has been the biggest frustration. I tried the "Field usage stats API" but there were always fields missing. Using the "Get field mapping API" returns everything in the mapping which is way too broad. How does Kibana know which fields are available for each index (the one shown under "available fields"?
These are the lines of code I used to pull the fields from each index.
field_usage_stats = es.indices.field_usage_stats(index=index) #too few fields returned
field_mapping = es.indices.get_mapping(index=index) #too many fields returned
Reference
https://elasticsearch-py.readthedocs.io/en/7.x/api.html
https://www.elastic.co/guide/en/elasticsearch/reference/7.16/field-usage-stats.html
https://www.elastic.co/guide/en/elasticsearch/reference/7.16/indices-get-mapping.html