Semaphore examples are lacking

[Darksonn]

Currently, the documentation for tokio::sync::Semaphore has two top-level examples with these titles:

1. Basic usage.
2. Use Semaphore::acquire_owned to move permits across tasks.

These examples aren't great. I would prefer to instead have examples that show common use-cases of semaphores. Here are some examples I thought of:

• Limit number of open files to 100 by acquiring a permit before opening a file, and dropping it after closing it.
• Limit number of incoming requests being handled at the same time, using acquire_owned. (Similar to current second example, but the title actually explains what the example does, rather than mentioning that it uses acquire_owned.)
• Prevent several tests from running at the same time, using a static semaphore with one permit. (This example can be reused both as a top-level example, and as the example for const_new.)
• A simple implementation of a token bucket for rate limiting, which uses add_permits and SemaphorePermit::forget.

This is a good first issue. Contact me for mentoring. I'm also open to other examples than the ones I mentioned; they're just my ideas. It's also perfectly okay to submit a PR that only partially resolves this issue, e.g. by adding just one example.

[andar1an]

May be too much for docs, but this is where I hit needing semaphore. Thanks again for help.

Docker Compose in Test Example

// For tests required to start the same infra to run
#[cfg(test)]
mod tests {
    use super::*;
    use std::time::Duration;
    use tokio::process::Command;
    use tokio::sync::{Semaphore, TryAcquireError};
    use tokio::time::sleep;
    use std::fs;

    static semaphore: Semaphore = Semaphore::const_new(1);
    
    #[tokio::test]
    async fn test_a() -> Result<(), anyhow::Error> {
        let infra_permit = semaphore.acquire().await.unwrap();
        
        let current_dir = std::env::current_dir()?;
        // where docker-compose.yaml is located
        let tests_dir = current_dir.join("tests/");
        let canonicalized_dir = fs::canonicalize(tests_dir)?;

        // Run docker-compose up
        let _docker_compose = Command::new("docker")
            .args(["compose", "up"])
            .current_dir(canonicalized_dir.clone())
            .spawn()?;

        // dk if there is better way to wait
        sleep(Duration::from_secs(5)).await;
        
        //Do Stuff

        let docker_compose_down = Command::new("docker")
            .args(["compose", "down"])
            .current_dir(canonicalized_dir)
            .output()
            .await?;

        drop(infra_permit);
        Ok(())
    }

    #[tokio::test]
    async fn test_b() -> Result<(), anyhow::Error> {
        let infra_permit = semaphore.acquire().await.unwrap();
        
        let current_dir = std::env::current_dir()?;
        // where docker-compose.yaml is located
        let tests_dir = current_dir.join("tests/");
        let canonicalized_dir = fs::canonicalize(tests_dir)?;

        // Run docker-compose up
        let _docker_compose = Command::new("docker")
            .args(["compose", "up"])
            .current_dir(canonicalized_dir.clone())
            .spawn()?;

        // dk if there is better way to wait
        sleep(Duration::from_secs(5)).await;

        //Do Stuff
        
        let docker_compose_down = Command::new("docker")
            .args(["compose", "down"])
            .current_dir(canonicalized_dir)
            .output()
            .await?;

        drop(infra_permit);
        Ok(())
    }

[Darksonn]

Yeah, that's a great starting point for the third example, but the examples generally need to be shorter.

[andar1an]

Ya, cutting out the docker part would be good. But I spent a lot of time troubleshooting std::process::Command being blocking. I included that in a comment because that may also be relevant to others trying to use a static semaphore for command processes (I imagine I can use std::process::Command now with the semaphore - sticking to tokio::process::Command though haha).

[alexanderkirilin]

I'm green to Tokio (and Rust in general) -- but I'll give this a shot.

[Darksonn]

@alexanderkirilin Make sure you don't submit the same example as in #5939. We don't want duplicate work.

[Darksonn]

@alexanderkirilin Which example did you intend to add?

[alexanderkirilin]

@Darksonn Hi, sorry for the delay. I made a PR for a spin on the second one (Limit number of incoming requests being handled at the same time, using acquire_owned. (Similar to current second example, but the title actually explains what the example does, rather than mentioning that it uses acquire_owned); here: #5956

[Darksonn]

The only missing example from my list is preventing several tests from running at the same time. Anyone wants to take it?

[songmuhan]

I'd like to take the last example. Can you provide more details about it? I'm curious, under what circumstances would we want to prevent several tests from running simultaneously? @Darksonn

[Darksonn]

For example, maybe the tests are talking to the same database, so you want to prevent them from interfering with each other.

[songmuhan]

I've drafted the example below and would appreciate any feedback on how to improve it, @Darksonn.
I chose to post it here instead of submitting a PR because I'm unsure about its quality. If I were a reader, I might question why two different keys weren't used to simplify things. 😂

/// ## Ensure Tests with Mutual Dependencies Run Sequentially
///
/// By default, Rust runs tests concurrently. However, in some scenarios, concurrent execution might lead to test interference.
/// For example, multiple tests might communicate with the same database, necessitating sequential execution to prevent data conflicts.
///
/// Consider the following scenario:
/// 1. `test_insert`: Inserts a key-value pair into the database, then retrieves the value using the same key to verify the insertion.
/// 2. `test_update`: After insertion, this test updates the key's value to a new one and verifies if the value has been accurately updated.
/// 3. `test_others`: This is a non-interfering test that can run concurrently with other tests.
///
/// For this example, it's essential that `test_insert` and `test_update` run sequentially, though their execution order is inconsequential.
/// Leveraging a semaphore with a single permit elegantly addresses this challenge.

#[cfg(test)]
mod tests {
    use tokio::sync::Semaphore;
    // only one permit can be acquired
    static PERMIT: Semaphore = Semaphore::const_new(1);
    // initilization of database
    static DB: database = database::setup();
    #[tokio::test]
    async fn test_insert() {
        // acquire permit first
        let permit = PERMIT.acquire().await.unwrap();
        let (key, value) = ("name", 0);
        DB.insert((key, value)).await;
        assert_eq!(DB.get(key), value);
        // undo the modification
        DB.delete(key).await;
        // permit dropped here
    }
    #[tokio::test]
    async fn test_update() {
        let permit = PERMIT.acquire().await.unwrap();
        let (key, value) = ("name", 0);
        DB.insert((key, value)).await;
        let new_value = 1;
        // update new value at the same key
        DB.update((key, new_value)).await;
        assert_eq!(DB.get(key).await, new_value);
        DB.delete(key).await;
    }
    #[tokio::test]
    async fn test_other() {
        // this test can simualtenously run with test_insert and test_update.
    }
}

[Darksonn]

It looks fine, go ahead and post a PR. There are some minor details I'd change, but it's easier for me to mention them using the PR interface, since it makes it easy to point at specific lines.

If I were a reader, I might question why two different keys weren't used to simplify things.

You can just say that using two different keys is preferable when possible, but there are cases where it is difficult to separate them like that, and in those cases you can use this strategy.

[songmuhan]

can you give a look? @Darksonn

[Darksonn]

Thanks everyone! The examples have been added. As a preview, check out the new documentation:

https://deploy-preview-6050--tokio-rs.netlify.app/tokio/sync/struct.semaphore

[andar1an]

This is absolutely wonderful. Amazing