openai command not found (mac)

Question:

I’m trying to follow the fine tuning guide for Openai here.

I ran:

pip install --upgrade openai

Which install without any errors.

But even after restarting my terminal, i still get

zsh: command not found: openai

Here is the output of echo $PATH:

/bin:/usr/bin:/usr/local/bin:/Users/nickrose/Downloads/google-cloud-sdk/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin

Here is the output of which python:

/usr/bin/python

Any tips for how to fix this? I’m on MacOS Big Sur 11.6.

Asked By: Nick

||

Answers:

Basically pip installs the packages under its related python directory, in a directory called site-packages (most likely, I’m not a python expert tbh). This is not included in the path you provided. First, ask pip to show the location to the package:

pip show openai

The output would be something like this:

Name: openai
Version: 0.22.0
Summary: Python client library for the OpenAI API
Home-page: https://github.com/openai/openai-python
Author: OpenAI
Author-email: [email protected]
License: 
Location: /Users/<USER>/DIR/TO/SOME/PYTHON/site-packages
Requires: numpy, openpyxl, pandas, pandas-stubs, requests, tqdm
Required-by:

So your package will be available in

/Users/<USER>/DIR/TO/SOME/PYTHON/site-packages/openai

Either add /Users/<USER>/DIR/TO/SOME/PYTHON/site-packages/ to your path, or use the complete address to your package, or try to access it using your python:

python -m openai # -m stands for module

To get more information about the -m flag, run python --help.

Update

So as you mentioned in the comments, you get permission denied after you add the directory to your package. This actually means that the package exists, but it’s not permitted by your OS to execute. This is the thing you have to do, locate your package, and then:

sudo chmod +x /PATH/TO/script

And the reason you’re getting command not found after you use sudo directly with the package, is that you update your path variable in zsh, but when you use sudo, superuser uses sh instead of zsh.

Answered By: AminMal

So what happens is that after installing the package there are no actual executables available. That’s why you get the error message when you try to execute for example:

openai --help

What i managed to find is that the actual parsing of the commands is done in

/Users/<USER>/DIR_TO_PYTHON/site-packages/openai/_openai_scripts.py

That’s just a python script which by default is not executable, so you have to make a workaround of which I find the easiest is creating an executable which basically calls it with the given arguments. Below are the steps which I’ve done to make it work on "macOS Monterey 12.0.1"

Locate the "openai" package which should be in

/Users/<USER>/DIR_TO_PYTHON/site-packages/

Make sure you are in the "openai" package folder and run

sudo vim /bin/openai

That should create a new file, put in the following command and make sure the path to the file is correct

python3 /Users/<USER>/DIR_TO_PYTHON/site-packages/openai/_openai_scripts.py $@

$@ is for the params that you pass when you call the executable

After saving the file the next step is making it executable which is done with

chmod +x /bin/openai

Last step is adding it to the PATH which is done by adding the file path in /etc/paths and after restarting the terminal, you should have fully working openai command globally

Answered By: Veselin Yotov

This doesn’t answer the question directly but specifies an alternative if you only want to prepare the data set and create the new model for finetunning. It doesn’t matter which system you have.

After a lot of struggle I decided it was not worth the hassel to run the cli on my specific machine because of so many different configurations and the mess. My end goal was just to create a model and upload it to OpenAI.

So if someone else stumbles on this post, just use Google Colab. I have also shared one of mine with steps to follow in here.

In case the links don’t work in the future I’ll list the steps here below as well:

(Step 1)

Set your API key (The already added api key is fake so please replace it with your own):

%env OPENAI_API_KEY=sk-Kz8Weh1234ddgYBmsdfinsdf7ndsfg55532432

(Step 2)

Install the openai package with pip like the following:

!pip install -Uq openai

(Step 3)

Import the openai package like the following:

import openai

(Step 4)

Make sure to upload the promptdata.csv file in the Google Colab folders.

The way to do it is:

  1. On the right side you’ll see a Hamburger Menu icon click on it.
  2. You’ll see the "Table of Contents"
  3. Click on the last folder icon on the top. If you hover on the icon it says "Files".
  4. Now you’ll see a folder called "sample_data".
  5. Click on the three dots menu for "sample_data" and then select "upload".
  6. You should be able to upload your csv file
  7. It is not mandatory to upload a csv file. You can also upload any type of TSV, XLSX, JSON or JSONL file as listed by the OpenAI documentation here. But it will always be converted to JSONL file after runnning the below command.

Once you’re done uploading the file you can run the below command to prepare your data set which will return you a new JSONL file at the same location where the original file was with all the corrections the tool provides.

!openai tools fine_tunes.prepare_data -f "/content/sample_data/promptdata.csv"

(Step 5)

Run the below command once again after the corrections and it will most likely say "No remediations found".

!openai tools fine_tunes.prepare_data -f "/content/sample_data/promptdata_prepared.jsonl"

(Step 6)

Finally run the below command using the file promptdata_prepared.jsonl and create a model.

!openai api fine_tunes.create -t "/content/sample_data/promptdata_prepared.jsonl"

(Step 7)

Once the model is created note the name of the "Uploaded model"

Answered By: Clive Machado

I was facing similar issue. It might due to global python in your machine is not maching with the pip installation path and it might be installing in some other python folder like in 3.9 and you have 3.10 python version globally set in your Mac.

First install fresh python using homebrew

brew install python

It will install the latest python into your machine. Then try to install openai again using

pip3 install openai

OR using pip (you can try installing using both and see which works as per your system config)

pip install openai

Now

ENJOY a cup of coffee ;)
Answered By: Sonu Kumar

I encountered a similar error, but after some searching, I successfully installed it with brew

brew install openai

install openai

Answered By: nguyên

This is what worked for me:

I created an executable in /usr/local/bin that executes the OpenAI Python script:

~/.local/lib/python3.10/site-packages/openai/_openai_scripts.py

Here’s the command I used:

 sudo sh -c 'echo python3 /home/<MY_USERNAME>/.local/lib/python3.10/site-packages/openai/_openai_scripts.py $@ > /usr/local/bin/openai'
Answered By: Shamus Cooley

After trying several other suggestions which didn’t work for me on my Mac, I consulted my son who is a python developer. He suggested creating a python virtual environment on my Mac (MacOS: Ventura, 13.4; Chip: M1 Pro) as follows:

python3 -m venv openaiwork (this will create a new openaiwork virtual directory in the current location)

source openaiwork/bin/activate

pip install openai

I also needed to install pandas in this new virtual environment using:

pip install pandas

Once set up as described above, the openai scripts worked like a charm!

Answered By: Steve Lacy

If you meet:
WARNING: The script openai.exe is installed in ‘C:Users……local-packagesPython39Scripts’ which is not on PATH.
You should:
export PATH="~……local-packagesPython39Scripts:$PATH"

For MacOS and Linux:
export PATH="/path/to/scripts:$PATH"
For Windows:
setx /m PATH "%PATH%;C:pathtoscripts"
Replace C:pathtoscripts with path to scripts

Answered By: Wan
Categories: questions Tags: , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.