Query data in MongoDB by date (timestamp) using python Question: I am trying to query data by timestamp in a MongoDB using python, but I’m receiving nothing. I tried to query using other keys (houseColor or houseId ) and it worked, but when I tried to query by timestamp It did not work. Am I …
Subtract dates on different lags Question: I have a dataset with two columns, Date1 and Date2. My goal is to subtract date2 from date1 at lag 0, lag 1, lag 2, …, lag 6. If date2 is bigger, then flag it as 1 otherwise 0. I need to do this iteratively so date2[0] – date1[0] …
Special case of counting empty cells "before" an occupied cell in Pandas Question: Pandas question here. I have a specific dataset in which we are sampling subjective ratings several times over a second. The information is sorted as below. What I need is a way to "count" the number of blank cells before every "second" …
Pandas dataframe – find last timestamp with valid values Question: I have a pandas dataframe in which the index is the timestamp and I have a column that contains a value per timestamp, like this: Values timestamp 2022-03-17 13:21:00+00:00 15.2 2022-03-22 13:24:00+00:00 17.8 2022-03-27 13:27:00+00:00 NaN 2022-03-30 13:30:00+00:00 NaN In the column of Values sometimes …
Checking pandas Timestamp for timezone string Question: In the code below I want to acquire the timezone of a pandas Timestamp supplied to a function: import pandas as pd import pytz timestamp = pd.Timestamp("2022-05-01", tz=’Europe/Paris’) print(timestamp.tzinfo) This prints: Europe/Paris However, I would like to check whether the timezone matches the timezone that I want and …
Timestamp for the API request Question: I am asking if I can link an API request with a timestamp that it sent so I can sort them later because I am sending them concurrently and sometimes they come out of order. here is my code and I am appending the results I wounder if I …
Subtract last timestamp from first timestamp for each Id in Pandas Dataframe Question: I have a dataframe (df) with the following structure: retweet_datetime tweet_id tweet_datetime 2020-04-24 03:33:15 85053699 2020-04-24 02:28:22 2020-04-24 02:43:35 85053699 2020-04-24 02:28:22 2020-04-18 04:24:03 86095361 2020-04-18 00:06:01 2020-04-18 00:19:08 86095361 2020-04-18 00:06:01 2020-04-18 00:18:38 86095361 2020-04-18 00:06:01 2020-04-18 00:07:08 86095361 2020-04-18 00:06:01 …
How do I convert a column to Pandas Timestamps? Question: I have a column in my DataFrame with values like ‘2022-06-03T00:00:00.000Z’ and I want to convert these (in place) to pd.Timestamp. I see many answers he on how to convert to np.datetime64 and on how do convert arbitrary columns of DataFrames, but can’t figure out …
convert time to UTC in pandas Question: I have multiple csv files, I’ve set DateTime as the index. df6.set_index("gmtime", inplace=True) #correct the underscores in old datetime format df6.index = [" ".join( str(val).split("_")) for val in df6.index] df6.index = pd.to_datetime(df6.index) The time was put in GMT, but I think it’s been saved as BST (British summertime) …
Add minutes from another column to string time column in pyspark Question: I have the below pyspark dataframe.both are string columns. time additional_time_in_mins 11:00:00 60 13:00:00 60 14:00:00 30 I have to add the minutes in the additional time column to actual time and create an output as below in pyspark. Expected output: new_time 12:00:00 …
