Looking deeper into this issue it appears that both PeriodicExportingMetricReader and BatchSpanProcessor shutdown methods first perform self._daemon_thread.join and only then notify child loops like self._exporter.shutdown which I believe is incorrect usage of threading.

Finally, OTLPExporterMixin._export method has this logic in place

        max_value = 64
        # expo returns a generator that yields delay values which grow
        # exponentially. Once delay is greater than max_value, the yielded
        # value will remain constant.
        for delay in _expo(max_value=max_value):

which basically means that even if we notify the exporter loop to shutdown it can hit the spot where it sleeps for 1 minute and only then would become aware of the event.

Possible solution could be to add exporter shutdown event and a slightly customized sleep method that would be aware of that event

    def sleep(self, delay: int) -> None:
        slept = 0
        while (slept < delay) and not self._shutdown_event.wait(1):
            slept += 1

something like this ^

And finally changing the order in which shutdown happens where it would first notify the exporter(s) and then listen for the thread to die (.join())

I have a similar issue - some of the users of our application run it on systems with the required port blocked by firewalls. The code defines the UNAVAILABLE error as transient, and thus keeps retrying on the blocked port. I would request that there be a setting to not retry on StatusCode.UNAVAILABLE, because it makes no sense in this case.

The same for me.
Agreed with @Elli-Rid , method's invokes inside BatchSpanProcessor.shutdown should be swapped to

self.span_exporter.shutdown()
self.worker_thread.join()

Another change is about OTLPSpanExporter._export. It's better to change time.sleep on threading.Event.wait(delay)t o not wait the last retry which can lasts up to 64 seconds. Smth like that:

class OTLPSpanExporterExtended(OTLPSpanExporter):
    def __init__(self, *args, **kwargs):
        self.shutdown_event = threading.Event()
        super().__init__(*args, **kwargs)

    def shutdown(self) -> None:
        super().shutdown()
        self.shutdown_event.set()
  
    def _export(self, *args, **kwargs):
        ...
        self.shutdown_event.wait(delay)
        ...

@Elli-Rid seems like you almost have a solution, do you think you could open a PR for this issue? 🙂

Kindly review my PR. I hope shutting down exporters before calling threads.join(), should resolve the problem.
Users shouldn't have to wait then.

I have a bunch of additional context in #2663, but I'm not sure if it's all still relevant. @rajat315315 your PR looks valuable but I think the biggest issue was called out by @Elli-Rid above #3309 (comment): the exponential backoff is unconditionally sleeping

@rajat315315 could you make a separate PR with this?

@aabmass I have updated my PR with waiting for a shutdown_event instead of unconditional sleep..