How to compare one set of data to another and choose the smallest degree value?

Well, here are some pointers towards a mechanical answer to the question. It feels unsatisfying because I still don’t fully understand what the higher objective function is that we want to optimize.

My understanding so far is that a df is given, containing a time series of points in 3 dimensions. The OP wants to find two contiguous and distinct subsets of df (defined by two non-overlapping intervals of time), such that some measure of dissimilarity between the two subsets is minimized. These two subsets are two clouds of points. Two metrics are used:

1. the direction of minimum variance of the combined clouds, obtained by PCA (eigensystem of the covariance matrix of all points);
2. the direction orthogonal to the vector difference between the clouds’ barycenters, and coplanar with (origin, mean_1, mean_2). Since the problem is 3-dimensional, the OP uses cross-products to calculate this.

The measure to minimize is the angle between these two metrics. No intuitive explanation as to why that is desirable has been provided at the time of this writing. In addition to the lack of intuition, there are caveats to both metrics (see the Discussion section below).

In any case, the following is a brute-force answer. It will take forever on any reasonable size dataset. It examines every pair of intervals satisfying the OP’s constraints and selects the pair of intervals for which the measure is minimum. The constraints are that the two intervals are each drawn from bigger, non-verlapping ones that are given.

Some code cleanup

import pandas as pd
import numpy as np

def select(df, t0, t1, t2, t3):
    df1 = df.truncate(before=t0, after=t1)
    df2 = df.truncate(before=t2, after=t3)
    return df1, df2

def eigen(df1, df2):
    data = pd.concat([df1, df2], axis=0).values.T
    covMatrix = np.cov(data, bias=True)
    eigval_mva, eigvec_mva = np.linalg.eig(covMatrix)
    return eigval_mva.real, eigvec_mva

def norm(v):
    return v / np.linalg.norm(v)

def crossdir(df1, df2):
    r1 = df1.mean().values
    r2 = df2.mean().values
    v = np.cross(r1 - r2, np.cross(r1, r2))
    return norm(v)

def angle(v1, v2):
    dp = np.dot(norm(v1), norm(v2))
    return np.arccos(dp)

def measure(df1, df2):
    va, ve = eigen(df1, df2)
    vmin = ve[va.argmin()]  # eigenvector with minimal eigenvalue
    nor = crossdir(df1, df2)
    return angle(vmin, nor)

A brute force approach

Edit: from discussion in comments, it appears that a constraint can be exploited: the two intervals have to be drawn from two larger intervals that are given.

from itertools import combinations, product
from math import comb
from tqdm.notebook import tqdm

def n_intervals(df, t0, t1, t2, t3):
    n1 = df.truncate(before=t0, after=t1).shape[0]
    n2 = df.truncate(before=t2, after=t3).shape[0]
    return comb(n1, 2) * comb(n2, 2)

def gen_intervals(df, t0, t1, t2, t3):
    ix1 = df.truncate(before=t0, after=t1).index
    ix2 = df.truncate(before=t2, after=t3).index
    for (t0, t1), (t2, t3) in product(
        combinations(ix1, 2), combinations(ix2, 2)):
        yield t0, t1, t2, t3

def brute(df, t0, t1, t2, t3):
    best = float('inf')
    best_ix = None
    nc = n_intervals(df, t0, t1, t2, t3)
    pb = tqdm(gen_intervals(df, t0, t1, t2, t3), total=nc)
    for ix in pb:
        df1, df2 = select(df, *ix)
        y = measure(df1, df2)
        if y < best:
            best = y
            best_ix = ix
            ixstr = [f'{t:%H:%M}' for t in ix]
            pb.set_description(f'y={y:4g} for ix={ixstr}')
    return best_ix, best

Small reproducible example

n = 4  # knots
np.random.seed(0)
df = pd.DataFrame(
    np.random.uniform(-10, 10, (n, 3)),
    index=pd.date_range('2020-07-07', freq='10min', periods=n),
    columns=['r1', 'r2', 'r3']
)
df = df.resample('min').interpolate('cubic')

Then:

# "big intervals" that should contain the small ones
interval0 = '2020-07-07 00:00:00', '2020-07-07 00:10:00'
interval1 = '2020-07-07 00:20:00', '2020-07-07 00:30:00'

# brute force exploration
ix, y = brute(df, *interval0, *interval1)

# takes ~5 sec for a df of 31 points

>>> ix
(Timestamp('2020-07-07 00:01:00', freq='T'),
 Timestamp('2020-07-07 00:02:00', freq='T'),
 Timestamp('2020-07-07 00:27:00', freq='T'),
 Timestamp('2020-07-07 00:28:00', freq='T'))

>>> y
0.1649...

Plot df, the two "big" intervals to chose from, and the two intervals found:

ax = df.plot()
ax.axvspan(*interval0, color='grey', alpha=0.1)
ax.axvspan(*interval1, color='grey', alpha=0.1)
ax.axvspan(ix[0], ix[1], color='grey', alpha=0.3)
ax.axvspan(ix[2], ix[3], color='grey', alpha=0.3)

Discussion

We need more information about the meaning of the measure being minimized. Not only is it very possible that drastically more efficient optimization techniques can be applied, the actual validity of the measure is called in question.

The meaning of the measure calculated is important and deserves scrutiny. Additionally, we should note conditions by which the measure can be unstable and yield unexpected results. For example, the axis of minimum variance (found by PCA) can be arbitrarily unstable: a close to spherical cloud of points (more precisely, one with close to spherical inertia) will yield a random direction that can be completely affected by as little as a single point added or removed.

Likewise, the direction orthogonal to the two barycenters can be arbitrarily unstable if the two are too close to each other.