Old post, but in case other people are looking, you can do the auto-shrink with something like this code snippet

function PopupEditor_ForceResize() {
    try {
        var popupDiv = window.top.$('.os-internal-ui-dialog-content');
        var popupIframe = window.top.$('.os-internal-ui-dialog-content iframe:first');
        var popupIframeOldHeight = popupIframe.height();
        
        // reduce iframe height to its contents
        var mainPopupHeight = document.getElementsByClassName('MainPopup')[0].clientHeight;
        popupDiv.height(mainPopupHeight);
        
        // pop-up editor already auto-grows, so we just need to address auto-shrink
        if (popupIframe.height() >= popupIframeOldHeight) {
            return false;
        }
        
        var result = window.top.RichWidgets_Popup_Editor_resize(popupDiv, -1, -1, false, {target: popupDiv});
        
        if (!result) {
            // something wrong happened in resize - reset iframe height
            popupIframe.height(popupIframeOldHeight);
        }
    } catch (err) {}
}

Usage:

Place that JS on your webscreen, then in a server action where you think your popup content can shrink, do a HTTPRequestHandler/RunJavaScript and call the action 

"PopupEditor_ForceResize();"

That code is from https://success.outsystems.com/Documentation/Development_FAQs/How_to_resize_a_pop-up_window

Happy OSing!