How to add/ modify/ patch altair chart data dynamically?

I would like to add chart(s), or data to existing chart dynamically (via ipywidget.interact) as in the code below (chart + dotchart). I get nearly what I want except whole chart gets re-drawn and this causes flickering.

How do I add/ modify/ patch data dynamically and avoid re-drawing whole chart?

Thanks!

import pandas as pd
import numpy as np
import altair as alt
from ipywidgets import interact

df = pd.DataFrame({"xval": range(100), "yval": np.random.randint(0,100,100)})
chart = alt.Chart(df).mark_point().encode(x="xval", y="yval",)

def update(x, y):
    dot = pd.DataFrame(dict(x=[x], y=[y]))
    dotchart = alt.Chart(dot).mark_point().encode(x="x", y="y", color=alt.value("red"))
    return chart + dotchart 

interact(update, x=(0, 100), y=(0, 100))
# x, y widgets that control position of 'red dot'