The onsuccess runs if ajax request itself was successful (i.e. there's no network error, uncaught exception, etc), not if action method was successfully invoked.

Given a <p:dialog widgetVar="testDialog">, you could remove the onsuccess and replace it by PrimeFaces RequestContext#execute() inside saveMethod():

if (success) {
    RequestContext.getCurrentInstance().execute("PF('testDialog').hide()");
}

Note: PF() was introduced in PrimeFaces 4.0. In older PrimeFaces versions, you need testDialog.hide() instead.

If you prefer to not clutter the controller with view-specific scripts, you could use oncomplete instead which offers an args object which has a boolean validationFailed property:

<p:commandButton ...
    oncomplete="if (args &amp;&amp; !args.validationFailed) PF('testDialog').hide()" />

The if (args) check is necessary because it may be absent when an ajax error has occurred and thus cause a new JS error when you try to get validationFailed from it; the & instead of & is mandatory for the reason explained in this answer, refactor if necessary to a JS function which you invoke like oncomplete="hideDialogOnSuccess(args, testDialog)" as shown in Keep <p:dialog> open when validation has failed.

If there is however no validation error and the action method is successfully triggered, and you would still like to keep the dialog open because of e.g. an exception in the service method call, then you can manually trigger validationFailed to true from inside backing bean action method by explicitly invoking FacesContext#validationFailed(). E.g.

FacesContext.getCurrentInstance().validationFailed();

I've just googled up this solution. Basically the idea is to use actionListener instead of button's action, and in backing bean you add callback parameter which will be then check in button's oncomplete method. Sample partial code:

JSF first:

<p:commandButton actionListener="#{myBean.doAction}"
   oncomplete="if (!args.validationFailed &amp;&amp; args.saved) schedulesDialog.hide();" />

Backing bean:

public void doAction(ActionEvent actionEvent) {
    // do your stuff here...
    if (ok) {
        RequestContext.getCurrentInstance().addCallbackParam("saved", true);
    } else {
        RequestContext.getCurrentInstance().addCallbackParam("saved", false);
    }
}

Hope this helps someone :)

The easiest solution is to not have any "widget.hide", neither in onclick, neither in oncomplete. Remove the hide functions and just put

visible="#{facesContext.validationFailed}" 

for the dialog tag

I use this solution:

JSF code:

<p:dialog ... widgetVar="dlgModify" ... >
...
<p:commandButton value="Save" update="@form" actionListener="#{AdminMB.saveTable}" />
<p:commandButton value="Cancel" oncomplete="PF('dlgModify').hide();"/>

Backing bean code:

public void saveTable() {
    RequestContext rc = RequestContext.getCurrentInstance();
    rc.execute("PF('dlgModify').hide()");
}

I believe this is the cleanest solution. Doing this you don't need to change your buttons code. This solution overrides the hide function prototype.

$(document).ready(function() {
    PrimeFaces.widget.Dialog.prototype.originalHide = PrimeFaces.widget.Dialog.prototype.hide; // keep a reference to the original hide()
    PrimeFaces.widget.Dialog.prototype.hide = function() {
        var ajaxResponseArgs = arguments.callee.caller.arguments[2]; // accesses oncomplete arguments
        if (ajaxResponseArgs && ajaxResponseArgs.validationFailed) {
            return;  // on validation error, prevent closing
        }
        this.originalHide();
    };
});

This way, you can keep your code like:

<p:commandButton value="Save" oncomplete="videoDetalheDialogJS.hide();" 
   actionListener="#{videoBean.saveVideo(video)}" />

Using the oncomplete attribute from your command button and really simple script will help you a lot.

Your dialog and command button would be something similar to this:

<p:dialog widgetVar="dialog">
   <h:form id="dialogView">
       <p:commandButton id="saveButton" icon="ui-icon-disk"
           value="#{ui['action.save']}"
           update=":dataList :dialogView"
           actionListener="#{mbean.save()}"
           oncomplete="handleDialogSubmit(xhr, status, args)" />
   </h:form>
 </p:dialog>

An the script would be something like this:

<script type="text/javascript">
    function handleDialogSubmit(xhr, status, args) {
        if (args.validationFailed) {
            dialog.show();
        } else {
            dialog.hide();
        }
    }
</script>