In this scenario, you could create Universal Package instead of Nuget with the dll file.
Both Azure CLI and Azure Pipeline task are supported to publish and download Universal Package.
Universal Package task in Azure Pipeline: https://learn.microsoft.com/en-us/azure/devops/pipelines/artifacts/universal-packages?toc=%2Fazure%2Fdevops%2Fartifacts%2Ftoc.json&bc=%2Fazure%2Fdevops%2Fartifacts%2Fbreadcrumb%2Ftoc.json&view=azure-devops&tabs=classic
Choose the dll file as "Path to file(s) to publish"
