Provide Escape Routes
Don't force your users to be all-or-nothing with your SDK. A great example is with APIs that use access tokens of any kind; I work with a few that use JWTs. As a nice helper function to the user, the SDK will generate the JWT and send it with your request, which is great. The extra value is added when the user can also generate a JWT for their own use; you expose the functionality that the library uses so that developers can use it themselves if they want to.
If there's a particular piece of functionality that every consumer of the API is going to have to do, however small or well-documented, then add it to your SDK. I spent quite a lot of my life answering questions about how to handle a particular signature check to improve security on our APIs. I updated documentation, improved it, added examples .... once we put the feature into the SDKs, I don't think I've thought at all about this feature since!
Delightful but Unobtrusive
Keeping the presence of the SDK as a helper for a particular tech stack, but not a requirement, is a great aim here. SDKs give us tech-stack-aware functionality, and smoother experiences with autocomplete and supplementary dependencies included. But developers may well need to get away from the beaten track in ways that you can't anticipate so I'll always advocate adding small pieces but keeping the responsibility with the developer on choosing to mix in only the pieces that make sense for her project. Hopefully the examples here give you some ideas for your next library, too!