Increased Resource Usage

[rkosafo]

There appears to be an increase in the resources (memory and processor) used when editing and saving sql files. The behavior has been observed with with VS2017 and VSCode. In the case of VS, there is often the need to restart it when the memory increases and causes the machine to slow down.

Is this a known issue? Are there any workarounds?

[rspeele]

Can you describe the precise steps to repro?

How much is memory increasing on each save?

[rkosafo]

I'll try and monitor the changes. Usually, it goes up to over 1.5GB. Will report back with further details.

[rspeele]

Been thinking about this a bit more and I'm worried that the problem may be fundamental to it being a generative type provider. This involves generating a dynamic assembly and as far as I know, those can't be garbage collected unless they are created in a separate app domain which is then unloaded. I don't see any app domain creation code in the ProvidedTypes code or in the F# compiler so I suspect the generated assemblies never get freed.

If that's true, then every time you change your migration scripts all the types for your queries get regenerated and added to the memory usage of the compiler service. When you have a lot of SQL queries (which is absolutely an intended use of the type provider in my opinion) the incremental jump on each type-regeneration could be significant.

This has me thinking it may be best to make an early breaking change and switch to an erasing type provider. This would make it impossible to directly reference the generated types from C# or VB.NET, so you'd have to write a mapping layer to create real types for use in other languages, but practically speaking that layer will exist in a lot of apps anyway. Of course another big downside is that it would be a pain to rewrite the type provider to be erasing -- it's not a trivial change.

A significant upside of making this change would be that it would make it much easier to target .NET Core. I think in general erasing type providers are better supported since they are by far the majority in the F# ecosystem. It feels like generative type providers -- while I personally prefer them in concept -- are an afterthought.

[rkosafo]

The ability to use this from non F# languages is certainly a plus for having it as a regenerative type. However, if having it as erasing type will help fix the memory issues, then I would say that might be the way to go. I have currently used it for four applications (2 large - tsql, 2 small - sqlite) in production and this is not really an issue for the small ones.

The flexibility that Rezoom provides make it trivial to add additional scripts and this slows the process down. Particularly since the db is being designed with the application. Everything is fine however if there are very little changes to the script afterwards. It is not very ideal but certainly not a show stopper.

That said, if converting to erasing will help make the experience better, I think it will be the best way to go with it. I'm unfortunately not sure of how much a change this will be. Also, introducing breaking changes now might be ideal.

[rspeele]

I think erasing is the way to go forward. It will be a big change though and I can't say when I will get around to it. Thank you so much for your feedback and interest in this project!

…

On Sat, Sep 23, 2017 at 9:43 PM, rkosafo ***@***.***> wrote: The ability to use this from non F# languages is certainly a plus for having it as a regenerative type. However, if having it as erasing type will help fix the memory issues, then I would say that might be the way to go. I have currently used it for four applications (2 large - tsql, 2 small - sqlite) in production and this is not really an issue for the small ones. The flexibility that Rezoom provides make it trivial to add additional scripts and this slows the process down. Particularly since the db is being designed with the application. Everything is fine however if there are very little changes to the script afterwards. It is not very ideal but certainly not a show stopper. That said, if converting to erasing will help make the experience better, I think it will be the best way to go with it. I'm unfortunately not sure of how much a change this will be. Also, introducing breaking changes now might be ideal. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#16 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABj-WnToeVnWYpnNvVPymNaSr2UDXiqhks5slbPFgaJpZM4O8fdb> .