Python 2.7 special splunk object to dataframe conversion

Question

i am reading some data from splunk and i want to convert that format to pandas dataframe.

result = results.ResultsReader(service.jobs.export(SEARCH_COMMAND))
for rr in result:
    print(rr)

OrderedDict([('H', '123'), ('U', '[email protected]'), ('S', 'motv:SMP_SESSION_ID/1523537360524/-86840158')])
OrderedDict([('H', '456'), ('U', '[email protected]'), ('S', 'motv:SMP_SESSION_ID/1523537367876/-765151654')])
OrderedDict([('H', '145'), ('U', '[email protected]'), ('S', 'motv:SMP_SESSION_ID/1523537367571/540003017')])
OrderedDict([('H', '111'), ('U', '[email protected]'), ('S', 'motv:SMP_SESSION_ID/1523537376045/540216322')])
OrderedDict([('H', '222'), ('U', '[email protected]'), ('S', 'motv:SMP_SESSION_ID/1523537383484/-86104258')])

score 1 · Accepted Answer · answered Apr 12 '18 at 14:10

pandas.DataFrame accepts a list of dictionaries directly.

You can create a list via lst = list(result) and build the dataframe as below.

from collections import OrderedDict
import pandas as pd

lst = [OrderedDict([('H', '123'), ('U', '[email protected]'), ('S', 'motv:SMP_SESSION_ID/1523537360524/-86840158')]),
       OrderedDict([('H', '456'), ('U', '[email protected]'), ('S', 'motv:SMP_SESSION_ID/1523537367876/-765151654')]),
       OrderedDict([('H', '145'), ('U', '[email protected]'), ('S', 'motv:SMP_SESSION_ID/1523537367571/540003017')]),
       OrderedDict([('H', '111'), ('U', '[email protected]'), ('S', 'motv:SMP_SESSION_ID/1523537376045/540216322')]),
       OrderedDict([('H', '222'), ('U', '[email protected]'), ('S', 'motv:SMP_SESSION_ID/1523537383484/-86104258')])]

df = pd.DataFrame(lst)

print(df)

#      H                      U                                             S
# 0  123  [email protected]   motv:SMP_SESSION_ID/1523537360524/-86840158
# 1  456  [email protected]  motv:SMP_SESSION_ID/1523537367876/-765151654
# 2  145  [email protected]   motv:SMP_SESSION_ID/1523537367571/540003017
# 3  111  [email protected]   motv:SMP_SESSION_ID/1523537376045/540216322
# 4  222  [email protected]   motv:SMP_SESSION_ID/1523537383484/-86104258

Python 2.7 special splunk object to dataframe conversion

1 Answers1