Does torch.manual_seed include the operation of torch.cuda.manual_seed_all?
Question:
Does torch.manual_seed include the operation of torch.cuda.manual_seed_all?
If yes, we can just use torch.manual_seed to set the seed. Otherwise we should call both functions.
Answers:
See Pytorch lightning’s seed_everything:
random.seed(seed)
np.random.seed(seed)
torch.manual_seed(seed)
torch.cuda.manual_seed_all(seed)
Makes me believe these are all and only the required seeds.
Yes, torch.manual_seed() does include CUDA:
You can use torch.manual_seed() to seed the RNG for all devices (both CPU and CUDA):
Yes, torch.manual_seed calls torch.cuda.manual_seed_all internally.
Additional evidence other than @iacob‘s answer can be found in the PyTorch source code
def manual_seed(seed) -> torch._C.Generator:
...
if not torch.cuda._is_in_bad_fork():
torch.cuda.manual_seed_all(seed)
...
Does torch.manual_seed include the operation of torch.cuda.manual_seed_all?
If yes, we can just use torch.manual_seed to set the seed. Otherwise we should call both functions.
See Pytorch lightning’s seed_everything:
random.seed(seed)
np.random.seed(seed)
torch.manual_seed(seed)
torch.cuda.manual_seed_all(seed)
Makes me believe these are all and only the required seeds.
Yes, torch.manual_seed() does include CUDA:
You can use
torch.manual_seed()to seed the RNG for all devices (both CPU and CUDA):
Yes, torch.manual_seed calls torch.cuda.manual_seed_all internally.
Additional evidence other than @iacob‘s answer can be found in the PyTorch source code
def manual_seed(seed) -> torch._C.Generator:
...
if not torch.cuda._is_in_bad_fork():
torch.cuda.manual_seed_all(seed)
...