Centre county Pennsylvania, USA | The screen print below shows an example of data wrangling with pandas + python.
It shows that elapsed time for data read and trimmed parse of a spread sheet .csv file with 5020 rows and 7 columns (35142 cells) was less than 13 milliseconds, excluding download and print time. Processor was AMD A10-6700 APU with 9GB of RAM. OS was Ubuntu 16.04 LTS. The .csv datafile contained 5020 market days of daily price history for SPY (S&P 500 ETF). It ran and returned renamed and trimmed variables using a single call to the custom python function named print_trimmed_columns, as follows:
CSV_file = 'spy_daily_price-history-12-13-2019.csv'
trimrange = [0,4]
df,credit,TIME,OPEN,HIGH,LOW,LAST,CHANGE,VOLUME = print_trimmed_columns(csv_file,trimrange)
################## SCREEN PRINT ##############################
====== RESTART: /home/don/Pandas/pandas_data_wrangle.py =========
datafile Downloaded from Barchart.com as of 12-13-2019 03:22pm CST
datafile info:
spy_daily_price-history-12-13-2019.csv:
dataframe info:
RangeIndex: 5021 entries, 0 to 5020
Data columns (total 7 columns):
Time 5021 non-null object
Open 5020 non-null float64
High 5020 non-null float64
Low 5020 non-null float64
Last 5020 non-null float64
Change 5020 non-null float64
Volume 5020 non-null float64
dtypes: float64(6), object(1)
memory usage: 274.7+ KB
list of values for rows 0 to 4 in each (renamed) column:
TIME: ['12/13/19', '12/12/19', '12/11/19', '12/10/19']
OPEN: [316.87, 314.43, 314.03, 313.82]
HIGH: [318.67, 317.99, 314.7, 314.55]
LOW: [316.02, 314.17, 313.439, 312.81]
LAST: [317.29, 317.13, 314.42, 313.53]
CHANGE: [0.16, 2.71, 0.89, -0.35]
VOLUME: [64972606.0, 96585800.0, 53521400.0, 53107100.0]
elapsed time, excluding download and print: 12.476 milliseconds |
Data wrangling with pandas + python
