community: Added new Utility runnables for NVIDIA Riva.

[rmkraus]

Please tag this issue with nvidia_genai

• Description: Added new Runnables for integration NVIDIA Riva into LCEL chains for Automatic Speech Recognition (ASR) and Text To Speech (TTS).
• Issue: N/A
• Dependencies: To use these runnables, the NVIDIA Riva client libraries are required. It they are not installed, an error will be raised instructing how to install them. The Runnables can be safely imported without the riva client libraries.
• Twitter handle: N/A

All of the Riva Runnables are inside a single folder in the Utilities module. In this folder are four files:

• common.py - Contains all code that is common to both TTS and ASR
• stream.py - Contains a class representing an audio stream that allows the end user to put data into the stream like a queue.
• asr.py - Contains the RivaASR runnable
• tts.py - Contains the RivaTTS runnable

The following Python function is an example of creating a chain that makes use of both of these Runnables:

def create(
    config: Configuration,
    audio_encoding: RivaAudioEncoding,
    sample_rate: int,
    audio_channels: int = 1,
) -> Runnable[ASRInputType, TTSOutputType]:
    """Create a new instance of the chain."""
    _LOGGER.info("Instantiating the chain.")

    # create the riva asr client
    riva_asr = RivaASR(
        url=str(config.riva_asr.service.url),
        ssl_cert=config.riva_asr.service.ssl_cert,
        encoding=audio_encoding,
        audio_channel_count=audio_channels,
        sample_rate_hertz=sample_rate,
        profanity_filter=config.riva_asr.profanity_filter,
        enable_automatic_punctuation=config.riva_asr.enable_automatic_punctuation,
        language_code=config.riva_asr.language_code,
    )

    # create the prompt template
    prompt = PromptTemplate.from_template("{user_input}")

    # model = ChatOpenAI()
    model = ChatNVIDIA(model="mixtral_8x7b")  # type: ignore

    # create the riva tts client
    riva_tts = RivaTTS(
        url=str(config.riva_asr.service.url),
        ssl_cert=config.riva_asr.service.ssl_cert,
        output_directory=config.riva_tts.output_directory,
        language_code=config.riva_tts.language_code,
        voice_name=config.riva_tts.voice_name,
    )

    # construct and return the chain
    return {"user_input": riva_asr} | prompt | model | riva_tts  # type: ignore

The following code is an example of creating a new audio stream for Riva:

input_stream = AudioStream(maxsize=1000)
# Send bytes into the stream
for chunk in audio_chunks:
    await input_stream.aput(chunk)
input_stream.close()

The following code is an example of how to execute the chain with RivaASR and RivaTTS

output_stream = asyncio.Queue()
while not input_stream.complete:
    async for chunk in chain.astream(input_stream):
        output_stream.put(chunk)    

Everything should be async safe and thread safe. Audio data can be put into the input stream while the chain is running without interruptions.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
langchain	⬜️ Ignored (Inspect)	Visit Preview		Feb 6, 2024 3:45am

[nfcampos]

I think these have to be awaited at the end, otherwise might be GCed before done

[rmkraus]

Good eye. Adding this now.

[haydeniw]

To use these runnables, the NVIDIA Riva client libraries are required. It they are not installed, an error will be raised instructing how to install them.
@rmkraus should we link to riva client installation docs? or the just the python library?

[rmkraus]

@haydeniw - This error is the error text if one were to try and use the Riva stages without the Riva Client python library:

Could not import the NVIDIA Riva client library.
Please install it with pip install nvidia-riva-client.

It is meant to be similar to the error message from the OpenAI Runnables.

[rmkraus]

Thanks @efriis! I made the necessary updates and used the provided make commands to ensure the tests ran for me locally. So everything is passing on Python 3.10 now. I think we can run again.

[rmkraus]

[rmkraus]

> cd libs/community
> rm -rf .venv
> poetry run python --version
Python 3.10.12
> poetry install --sync --with dev,lint,test -E extended_testing 
> make lint tests check_imports

That all passed locally. Should I have done something differently?

[rmkraus]

Looks like the issue was I had a call to the riva.client extended_testing dependency hiding. I used this to run tests to make sure to run w/ and w/out extended testing.

poetry install --sync --with dev,lint,test -E extended_testing; 
make lint tests check_imports;
poetry install --sync --with dev,lint,test; 
make lint tests check_imports;

@efriis: I'd like to say I think the tests should pass, but I think we can all agree I've done a pretty poor job intuiting that. But at least this issue has been resolved and tested on my side.

[nfcampos]

@rmkraus looks like a conflict has appeared

[rmkraus]

Resolving that conflict will be easy, resolving the last CI errors will be tougher....

After some digging:

• The DashVector integration requires dashvector 1.0.1 (Released Aug 24, 2023)
• dashvector 1.0.1 requires protobuf 3. Lastest version is 3.20.3. Release September 29, 2022. Supported ended 23Q2.
• The Riva Client library requires grpcio-tools. The latest version of grpcio-tools that supports Protobuf 3 is 1.48.2. This version is not compatible with Python 3.11.

Even the latest version of dashvector (1.0.10) requires Protobuf 3.

So, we would have to drop support for one of these things:

• dashvector
• Python 3.11
• Riva (this MR)

But holding Protobuf back to a version from 2022 seems risky.

@efriis: What is the best route to go with here?

[rmkraus]

Another potential solution, but it wouldn't be pretty.

The dashvector unit tests dont test the vector db integration and dont require the dashvector library. Also, the docs for using dashvector specifically say to install the dependency.

If dashvector is removed from the pyproject file, then the protobuf version can be bumped for all the other dependencies. A comment could be left in the pyproject file as to why it was commented out. All the unit tests will still pass, all the docs will still be valid, and no functionality is lost. Its just a bit ugly.

[efriis]

This actually works for me! Because dashvector isn't needed in any unit tests, it actually doesn't have to be part of the declared optional deps in pyproject.

[rmkraus]

Nobody sneeze! @efriis, all the tests and conflicts are resolved!

[efriis]

boom!