How would I make this whole program repeat after the person enters an answer? (tkinter)

simplest answer without making major changes in your script would be. You could get all the script inside a new function and call that function whenever you want to repeat showing window with circles and ask for input with frame ;

def call_window():
    number_of_circles = random.randint(1, 2)

    def new_circle(canvas):
        c1 = random.randint(1, 500)
        c2 = random.randint(1, 500)
        canvas.create_oval(c1, c2, c1 + 45, c2 + 45, fill='black')

    def enter_text():

        answer = int(text.get())
        if answer == number_of_circles:
            frame.destroy()
            label = Label(window, text="That's right")
            label.pack()

        else:
            frame.destroy()
            label = Label(window, text="That's wrong")
            label.pack()

    window = Tk()

    canvas = Canvas(window, width=1000, height=1000)



    for i in range(number_of_circles):
        new_circle(canvas)

    canvas.pack()

    window.after(750, canvas.destroy)

    frame = Frame()
    text = Entry(frame, width=50)

    text.pack()
    text.focus()

    enter = Button(frame, command=enter_text)
    enter.pack()
    frame.pack()




    window.bind('<Return>', enter_text)


    window.mainloop()

call_window()
call_window()

With above script whenever you do call_window() it will call your screen with circles and then ask for input.

Then you with simple while loop you can keep asking for user input and depending on that you can call_window():

is_on = True

while is_on:
    user = input("Do you want to continue? Y/N: ").lower()

    if user == "y":
        call_window()
    else:
        break

With this way after user clicks button, has to close frame window. To improve GUI behaviour and User Experience I believe object oriented tkinter window would be better ; https://www.pythontutorial.net/tkinter/tkinter-object-oriented-window/#:~:text=Defining%20a%20Tkinter%20object-oriented%20window&text=How%20it%20works.,to%20display%20the%20root%20window.

I also made a major change to improve, and accordingly, this rule applies to Python 3.5 and later(Walrus). I also removed the trap try/except. I used double space and single space in functions. Btw, for constant. There will be no extra whitespace. I added asyncio. Must improve improve upon @Sramazoth.

Here is code:

from tkinter import *
import random
import time
import asyncio


DIMENSIONS = [500, 500]
MAX_NB_CIRCLES = 5
SIZE_CIRCLE = 45

class ReflexApp(Tk):
    number_of_circles = None
    frame  = None
    text = None
    canvas = None
    label  = None

    def __init__(self, *args, **kwargs):
        Tk.__init__(self, *args, **kwargs)
        self.wm_geometry(str(DIMENSIONS[0])+"x"+str(DIMENSIONS[1]))
        self.bind('<Return>', self.binded_enter_text)
        self.new_circles()

        
    def new_guess(self, error):
        if not error:
            self.frame = Frame()
            self.text = Entry(self.frame, width=50)
            self.text.pack()
            self.text.focus()
            self.enter = Button(self.frame, text='Validate', command=self.enter_text)
            self.enter.pack()
            self.frame.pack()
        else:
            False
            

    def new_circles(self):
        self.canvas = Canvas(self, width=DIMENSIONS[0], height=DIMENSIONS[1])
        self.number_of_circles = random.randint(1, MAX_NB_CIRCLES)
        for i in range(self.number_of_circles):
            c1 = random.randint(1, DIMENSIONS[0]-SIZE_CIRCLE)
            c2 = random.randint(1, DIMENSIONS[1]-SIZE_CIRCLE)
            self.canvas.create_oval(c1, c2, c1+SIZE_CIRCLE, c2+SIZE_CIRCLE, fill='black')
        self.canvas.pack()
        self.after(750, self.canvas.destroy)
        self.new_guess(False)


    def binded_enter_text(self, event):
        self.enter_text()


    def enter_text(self):
        (answer := int(self.text.get()))
        if answer == self.number_of_circles:
            self.frame.destroy()
            self.label = Label(self, text="That's right")
            self.label.pack()           
        else:
            self.frame.destroy()
            self.label = Label(self, text="That's wrong")
            self.label.pack()
        self.after(750, self.new_circles)
        self.after(750, self.label.destroy)


async def main() -> None:
    my_app=ReflexApp()
    my_app.mainloop()

asyncio.run(main())