Having three output features

[shamsa00]

Hi and thanks for developing DeePyMoD. I have defined two inputs and three features for the output. The NN was defined as NN = NN(2, [30, 30, 30, 30], 3) then I got an error for scikit-learn: 'For multi-task outputs, use MultiTaskLassoCV'. Then I defined the estimator as estimator=Threshold( threshold=0.5,estimator=MultiTaskLassoCV()) , then I got the following error, 'scalar should be 0D' (torch/utils/tensorboard/summary.py). Could you please help me with that?
By the way, I tried to modify 'ODE_Example_coupled_nonlin.ipynb' so the differential equation has three variables and I modified the constructed library as well.

[GJBoth]

Hi,

I'm on holiday so I don't have too much time to look into this now, but the error is with the tensorboard logging. Somewhere it expects a scalar for writing (look for a function add_scalar), but you're giving it a vector, so check where that's happening. Hope this helps, if not, I'll fix it in two weeks.

[shamsa00]

Thank you very much for your answer, I guess it arises from defining the library. In this interval, I will try to fix it. , Am Tue, 31 Aug 2021 06:09:41 -0700 schrieb Gert-Jan Both ***@***.***>:

…

Hi, I'm on holiday so I don't have too much time to look into this now, but the error is with the tensorboard logging. Somewhere it expects a scalar for writing (look for a function `add_scalar`), but you're giving it a vector, so check where that's happening. Hope this helps, if not, I'll fix it in two weeks. -- You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub: #67 (comment)

[GJBoth]

Hi @shamsa00 , did you manage to fix the error? If not, can you show me a minimal script required to reproduce the error?

[shamsa00]

Hi, Thank you very much for your message. Yeah actually, I could fix it. The problem was that I have changed the following code: time_deriv = dy[:, 0:1] in building the library but after that it works properly for multiple input and output. Now, I have another question, could I apply group lasso in the regularization? Thanks in advance Am Wed, 15 Sep 2021 01:37:40 -0700 schrieb Gert-Jan Both ***@***.***>:

…

Hi @shamsa00 , did you manage to fix the error? If not, can you show me a minimal script required to reproduce the error? -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #67 (comment)

[GJBoth]

Great! Grouplasso should be possible. Pinging @georgestod , he's taken point on most of the group lasso stuff. Some of it isn't merged back into this repo yet but I think he can give you some pointers.

[georgestod]

Hi,
We have implemented a group Lasso but did not merge it back because it is still experimental. We are soon publishing about the promotion of grouped sparsity as there are better options than a bare group Lasso in terms in of model selection consistency. In the mean time, if you install (MuTaR - https://hichamjanati.github.io/mutar/ - this is a great package, I recommend it), a cross validated Group Lasso in DeepMod could look like this:

`class GroupLassoThresholdedFAST(GroupEstimator):

def __init__(self, threshold: float = 0.0, nfolds: int = 10, alphas: np.ndarray = np.logspace(-5, -1, 10, base=10), n_cores: int=-1, wm_s: bool=False) -> None:
    super().__init__()        
    self.threshold = threshold
    self.nfolds = nfolds
    self.alphas = alphas
    self.n_cores = n_cores
    self.wm_s = wm_s

def fit(self, X: np.ndarray, y: np.ndarray) -> np.ndarray:

    Args:
        X (np.ndarray): [description]
        y (np.ndarray): [description]
    Returns:
        np.ndarray: [description]

    
    # ---------------------------------------------------
    # Cross validated Group Lasso implementation
    # ---------------------------------------------------
    # cv parameters
    nfolds = self.nfolds
    alphas = self.alphas
    nalphas = len(alphas)

    # solver parameters
    tol_sol = 1E-5
    maxit = 1E4
    n_procs = self.n_cores
    wm_s   = self.wm_s
    
    # dimensions from data
    n_samples, n_features, n_exps = X[0].shape[0], X[0].shape[1], len(y)       
    
    # re-packing the data
    y_cv = y[0]
    X_cv = X[0]
    Xd = np.zeros([n_exps, n_samples, n_features])
    Xd[0,:,:] = X[0]
    for i in range(1,n_exps):
        y_cv = np.hstack([y_cv,y[i]])
        X_cv = np.hstack([X_cv,X[i]])
        Xd[i,:,:] = X[i]

    # auxiliary functions    
    def res_error(y,X,beta):
        # to compute a residual error
        xbeta = np.array([x.dot(th) for x, th in zip(X, beta.T)])
        return mean_squared_error(y,xbeta)

    def pred_error(X,beta,betahat):
        # to compute a prediction error (not used, but could help for debugguing)
        xbeta    = np.array([x.dot(th) for x, th in zip(X, beta.T)]) 
        xbetahat = np.array([x.dot(th) for x, th in zip(X, betahat.T)])
        return mean_squared_error(xbeta,xbetahat)
    
    def my_cross_val(train, test):
        # to parallelize cross validation using joblib, each core takes one fold of the CV            
        
        # splitting the data
        y_train = y_cv[train].T                                        # training set
        X_train = np.zeros([n_exps, y_cv[train].shape[0], n_features])
        y_test = y_cv[test].T                                          # test set
        X_test = np.zeros([n_exps, y_cv[test].shape[0], n_features])    
        for i in range(n_exps):
            X_train[i,:,:] = Xd[i,:,:][train]
            X_test[i,:,:] = Xd[i,:,:][test]
        # computing Group Lasso for varying sparsification parameters
        REs = []
        for alpha in alphas:
            # fitting on training set
            gl = GroupLasso(alpha=alpha,fit_intercept = False, max_iter = maxit, tol = tol_sol, warm_start=wm_s).fit(X_train, y_train)
            betahat = gl.coef_
            # scoring on test set
            REs.append(res_error(y_test,X_test,betahat))
        return REs
    
    # cross validation in parallel    
    kf = KFold(n_splits=nfolds)
    REs = Parallel(n_jobs=n_procs)(delayed(my_cross_val)(train, test) for train, test in kf.split(X_cv))
        
    # computing for minimum residual error at CV lambda using all data
    yl = np.array(REs).reshape([nfolds,nalphas]).T
    lambda_opt = alphas[np.argmin(np.mean(yl.T,axis=0))]
    gl = GroupLasso(alpha=lambda_opt,fit_intercept = False, max_iter=maxit, tol=tol_sol, warm_start=wm_s).fit(Xd, y_cv.T)
    betahat_opt = gl.coef_
    coeffs = betahat_opt
    # --------------------------------

    # group thresholding
    coeffs[np.all(np.abs(coeffs) < self.threshold, axis=1)] = 0.0

    # packing for transportation
    coeffs_list = [coeffs[:,i] for i in np.arange(n_exps)]
    
    return coeffs_list`

[shamsa00]

Thank you very much for both replies. Looking forward to seeing your new work with group sparsity inference.
I will try to implement the codes on DeepMod and back to you in case of further questions.

[georgestod]

Here is the new work in case you are interested !
https://arxiv.org/abs/2109.11939

[shamsa00]

Of course I am. Thank you very much. Am Tue, 28 Sep 2021 02:37:08 -0700 schrieb Georges Tod ***@***.***>:

…

Here is the new work in case you are interested ! https://arxiv.org/abs/2109.11939 -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #67 (reply in thread)