Rust BigInt memory allocation and performance compared to Python BigInt implementation

Question

When working with num_bigint::BigInt I have run into some performance limitations, for context I am working on https://www.geeksforgeeks.org/how-to-generate-large-prime-numbers-for-rsa-algorithm/ [full code at bottom of page].

In the nBitRand function we essentially shuffle a vector of BigInt’s determined by the range:

(2.pow(n - 1) + 1)..(2.pow(n) - 1)

I have made a function to calculate pow for BigInts:

fn big_pow(base: BigInt, exp: BigInt) -> BigInt
{
    if exp == BigInt::from_u32(0).unwrap()
    {
        return BigInt::from_u32(1).unwrap();
    }
    let mut answer = base.clone();
    let mut i = BigInt::from_u32(1).unwrap();
    while i < exp
    {
        i = i + BigInt::from_u32(1).unwrap();
        answer = answer * base.clone();
    }
    return answer;
}

Now here is the function for nBitRand which uses the big_pow function:

fn nBitRand(n: BigInt) -> BigInt
{
    let big2 = BigInt::from_u64(2).unwrap();
    let big1 = BigInt::from_u64(1).unwrap();
    let mut range: Vec<BigInt> =
        num_iter::range_inclusive
        (
            (big_pow(big2.clone(), n.clone() - big1.clone()) + big1.clone()),
            (big_pow(big2.clone(), n.clone()) - big1.clone())
        )
        .collect();
    range.shuffle(&mut rand::thread_rng());
    return range[0].clone();
}

When I try this function with a number n that is larger than 32 I get this panic:

memory allocation of 137438953440 bytes failed

Meanwhile the python implementation can do 1024 bits of n seamlessly?

Not to mention that if I do actually run this function for 32 bits of n ( in release mode or not )
it runs terribly slow and nearly crashes my PC which is running several other processes as most peoples PCs are.

Is this the same for every current BigInt implementation in rust? I would much rather use Rust than Python in general, is it expected that I should just use python for any number this big?

Asked By: Poseidon

||

Source

Answer 1

The reason you are having issues is that there are a lot of numbers between 2.pow(n - 1) and 2^pow(n), and your rust code is trying to hold all of them in memory at once. Just trying to hold the numbers between 2^31 and 2^32 in memory all at once will likely require a few tens of gigabytes of ram, which is evidently more than your computer can handle.

The correct way to handle this is to use something like the gen_bigint which handles this elegantly:

use rand::prelude::*;
use num::bigint::RandBigInt;
fn main(){
    println!("{}", thread_rng().gen_bigint(1024));
}

(no playground link because it requires activating the rand feature)

sample output:

-120678186469168050141127714278465650797509809181209645055716269782732917216510450905974158776294027244037250792198062938087103374539558073937169391547408613028913574776101294537137332842677479099864132223045958106561504255177421361770704902635830935432132237493695901107602818879400859307921127547090428474495

Answered By: Aiden4

Answer 2

If anyone wants to use this, add use num_bigint::RandBigInt; to your use directives.

Also in cargo.toml enable rand feature in bigint as such:

num-bigint = {version = "0.4.3", features = ["rand"] }

If you are working with num_bigint as I am.

Use the function like this:

gen_bigint_range(&mut rand::thread_rng(), &lowerbound.clone(), &higherbound.clone());

Docs: https://docs.rs/num-bigint/0.4.3/num_bigint/trait.RandBigInt.html#tymethod.gen_bigint_range

_{This answer was posted as an edit to the question Questions about Rust BigInt memory allocation and performance compared to Python BigInt implementation by the OP Poseidon under CC BY-SA 4.0.}

Answered By: vvvvv

Rust BigInt memory allocation and performance compared to Python BigInt implementation

Question:

Answers: