utils/hub

Utility functions to interact with the Hugging Face Hub (https://huggingface.co/models)

• utils/hub
  • static
    • .getFile(urlOrPath) ⇒ Promise.<(FileResponse|Response)>
    • .checkCachedResource(cache, localPath, proposedCacheKey) ⇒ Promise.<(Response|FileResponse|undefined|string)>
    • .storeCachedResource(path_or_repo_id, filename, cache, cacheKey, response, [result], [options]) ⇒ Promise.<void>
    • .loadResourceFile(path_or_repo_id, filename, [fatal], [options], [return_path], [cache]) ⇒ Promise.<(string|Uint8Array|null)>
      • ~cacheKey : string
      • ~response : Response | FileResponse | undefined | string
      • ~buffer : Uint8Array
    • .getModelFile(path_or_repo_id, filename, [fatal], [options], [return_path]) ⇒ Promise.<(string|Uint8Array)>
      • ~cache : CacheInterface | null
    • .getModelText(modelPath, fileName, [fatal], [options]) ⇒ Promise.<(string|null)>
    • .getModelJSON(modelPath, fileName, [fatal], [options]) ⇒ Promise.<Object>
  • inner
    • ~buildResourcePaths(path_or_repo_id, filename, [options], [cache]) ⇒ Object
    • ~ExternalData : boolean | number
    • ~PretrainedOptions : Object
    • ~ModelSpecificPretrainedOptions : Object
    • ~PretrainedModelOptions : PretrainedOptions | ModelSpecificPretrainedOptions

---

utils/hub.getFile(urlOrPath) ⇒ Promise. < (FileResponse|Response) >

Helper function to get a file, using either the Fetch API or FileSystem API.

Kind: static method of utils/hub
Returns: Promise.<(FileResponse|Response)> - A promise that resolves to a FileResponse object (if the file is retrieved using the FileSystem API), or a Response object (if the file is retrieved using the Fetch API).

Param	Type	Description
urlOrPath	URL | string	The URL/path of the file to get.

---

utils/hub.checkCachedResource(cache, localPath, proposedCacheKey) ⇒ Promise. < (Response|FileResponse|undefined|string) >

Checks if a resource exists in cache.

Kind: static method of utils/hub
Returns: Promise.<(Response|FileResponse|undefined|string)> - The cached response if found, undefined otherwise.

Param	Type	Description
cache	CacheInterface | null	The cache instance to check.
localPath	string	The local path to try first.
proposedCacheKey	string	The proposed cache key to try second.

---

utils/hub.storeCachedResource(path_or_repo_id, filename, cache, cacheKey, response, [result], [options]) ⇒ Promise. < void >

Stores a resource in the cache.

Kind: static method of utils/hub

Param	Type	Description
path_or_repo_id	string	The path or repo ID of the model.
filename	string	The name of the file to cache.
cache	CacheInterface	The cache instance to store in.
cacheKey	string	The cache key to use.
response	Response | FileResponse	The response to cache.
[result]	Uint8Array	The result buffer if already read.
[options]	PretrainedOptions	Options containing progress callback and context for progress updates.

---

utils/hub.loadResourceFile(path_or_repo_id, filename, [fatal], [options], [return_path], [cache]) ⇒ Promise. < (string|Uint8Array|null) >

Loads a resource file from local or remote sources.

Kind: static method of utils/hub
Returns: Promise.<(string|Uint8Array|null)> - A Promise that resolves with the file content as a Uint8Array if return_path is false, or the file path as a string if return_path is true.
Throws:

• Will throw an error if the file is not found and fatal is true.

Param	Type	Default	Description
path_or_repo_id	string		This can be either: a string, the model id of a model repo on huggingface.co. a path to a directory potentially containing the file.
filename	string		The name of the file to locate.
[fatal]	boolean	true	Whether to throw an error if the file is not found.
[options]	PretrainedOptions		An object containing optional parameters.
[return_path]	boolean	false	Whether to return the path of the file instead of the file content.
[cache]	CacheInterface | null		The cache instance to use.

• .loadResourceFile(path_or_repo_id, filename, [fatal], [options], [return_path], [cache]) ⇒ Promise.<(string|Uint8Array|null)>
  • ~cacheKey : string
  • ~response : Response | FileResponse | undefined | string
  • ~buffer : Uint8Array

---

loadResourceFile~cacheKey : string

Kind: inner property of loadResourceFile

---

loadResourceFile~response : Response | FileResponse | undefined | string

Kind: inner property of loadResourceFile

---

loadResourceFile~buffer : Uint8Array

Kind: inner property of loadResourceFile

---

utils/hub.getModelFile(path_or_repo_id, filename, [fatal], [options], [return_path]) ⇒ Promise. < (string|Uint8Array) >

Retrieves a file from either a remote URL using the Fetch API or from the local file system using the FileSystem API. If the filesystem is available and env.useCache = true, the file will be downloaded and cached.

Kind: static method of utils/hub
Returns: Promise.<(string|Uint8Array)> - A Promise that resolves with the file content as a Uint8Array if return_path is false, or the file path as a string if return_path is true.
Throws:

• Will throw an error if the file is not found and fatal is true.

Param	Type	Default	Description
path_or_repo_id	string		This can be either: a string, the model id of a model repo on huggingface.co. a path to a directory potentially containing the file.
filename	string		The name of the file to locate in path_or_repo.
[fatal]	boolean	true	Whether to throw an error if the file is not found.
[options]	PretrainedOptions		An object containing optional parameters.
[return_path]	boolean	false	Whether to return the path of the file instead of the file content.

---

getModelFile~cache : CacheInterface | null

Kind: inner constant of getModelFile

---

utils/hub.getModelText(modelPath, fileName, [fatal], [options]) ⇒ Promise. < (string|null) >

Fetches a text file from a given path and file name.

Kind: static method of utils/hub
Returns: Promise.<(string|null)> - The text content of the file.
Throws:

• Will throw an error if the file is not found and fatal is true.

Param	Type	Default	Description
modelPath	string		The path to the directory containing the file.
fileName	string		The name of the file to fetch.
[fatal]	boolean	true	Whether to throw an error if the file is not found.
[options]	PretrainedOptions		An object containing optional parameters.

---

utils/hub.getModelJSON(modelPath, fileName, [fatal], [options]) ⇒ Promise. < Object >

Fetches a JSON file from a given path and file name.

Kind: static method of utils/hub
Returns: Promise.<Object> - The JSON data parsed into a JavaScript object.
Throws:

• Will throw an error if the file is not found and fatal is true.

Param	Type	Default	Description
modelPath	string		The path to the directory containing the file.
fileName	string		The name of the file to fetch.
[fatal]	boolean	true	Whether to throw an error if the file is not found.
[options]	PretrainedOptions		An object containing optional parameters.

---

utils/hub~buildResourcePaths(path_or_repo_id, filename, [options], [cache]) ⇒ Object

Builds the resource paths and URLs for a model file. Can be used to get the resource URL or path without loading the file.

Kind: inner method of utils/hub
Returns: Object - An object containing all the paths and URLs for the resource.

Param	Type	Default	Description
path_or_repo_id	string		This can be either: a string, the model id of a model repo on huggingface.co. a path to a directory potentially containing the file.
filename	string		The name of the file to locate.
[options]	PretrainedOptions		An object containing optional parameters.
[cache]	CacheInterface | null		The cache instance to use for determining cache keys.

---

utils/hub~ExternalData : boolean | number

Specifies whether to load the model using the external data format.

• false: Do not use external data format
• true: Use external data format with 1 chunk
• number: Use external data format with the specified number of chunks

Kind: inner typedef of utils/hub

---

utils/hub~PretrainedOptions : Object

Options for loading a pretrained model.

Kind: inner typedef of utils/hub
Properties

Name	Type	Default	Description
[progress_callback]	ProgressCallback		If specified, this function will be called during model construction, to provide the user with progress updates.
[config]	PretrainedConfig		Configuration for the model to use instead of an automatically loaded configuration. Configuration can be automatically loaded when: The model is a model provided by the library (loaded with the model id string of a pretrained model). The model is loaded by supplying a local directory as pretrained_model_name_or_path and a configuration JSON file named config.json is found in the directory.
[cache_dir]	string	null	Path to a directory in which a downloaded pretrained model configuration should be cached if the standard cache should not be used.
[local_files_only]	boolean	false	Whether or not to only look at local files (e.g., not try downloading the model).
[revision]	string	"'main'"	The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a git-based system for storing models and other artifacts on huggingface.co, so revision can be any identifier allowed by git. NOTE: This setting is ignored for local requests.

---

utils/hub~ModelSpecificPretrainedOptions : Object

Options for loading a pretrained model.

Kind: inner typedef of utils/hub
Properties

Name	Type	Default	Description
[subfolder]	string	"'onnx'"	In case the relevant files are located inside a subfolder of the model repo on huggingface.co, you can specify the folder name here.
[model_file_name]	string	null	If specified, load the model with this name (excluding the .onnx suffix). Currently only valid for encoder- or decoder-only models.
[device]	DeviceType | Record.<string, DeviceType>		The device to run the model on. If not specified, the device will be chosen from the environment settings.
[dtype]	DataType | Record.<string, DataType>		The data type to use for the model. If not specified, the data type will be chosen from the environment settings.
[use_external_data_format]	ExternalData | Record.<string, ExternalData>	false	Whether to load the model using the external data format (used for models >= 2GB in size).
[session_options]	InferenceSession.SessionOptions		(Optional) User-specified session options passed to the runtime. If not provided, suitable defaults will be chosen.

---

utils/hub~PretrainedModelOptions : PretrainedOptions | ModelSpecificPretrainedOptions

Options for loading a pretrained model.

Kind: inner typedef of utils/hub

---

Update on GitHub