Using dlarray with betarnd/randg (2024)

22 views (last 30 days)

Show older comments

Jack Hunt on 20 Jun 2024 at 22:37

  • Link

    Direct link to this question

    https://www.mathworks.com/matlabcentral/answers/2130616-using-dlarray-with-betarnd-randg

  • Link

    Direct link to this question

    https://www.mathworks.com/matlabcentral/answers/2130616-using-dlarray-with-betarnd-randg

Commented: Jack Hunt about 22 hours ago

Accepted Answer: Matt J

Open in MATLAB Online

I am writing a custom layer with the DL toolbox and a part of the forward pass of this layer is making draws from a beta distribution where the b parameter is to be optimised as part of the network training. However, I seem to be having difficulty using betarnd (and by extension randg) with a dlarray valued parameter.

Consider the following, which works as expected.

>> betarnd(1, 0.1)

ans =

0.2678

However, if I instead do the following, then it does not work.

>> b = dlarray(0.1)

b =

1×1 dlarray

0.1000

>> betarnd(1, b)

Error using randg

Error in betarnd (line 34)

g2 = randg(b,sizeOut); % could be Infs or NaNs

Is it not possible to use such functions with parameters to be optimised via automatic differentiation (hence dlarray)?

Many thanks

0 Comments

Show -2 older commentsHide -2 older comments

Sign in to comment.

Sign in to answer this question.

Accepted Answer

Matt J on 20 Jun 2024 at 22:51

  • Link

    Direct link to this answer

    https://www.mathworks.com/matlabcentral/answers/2130616-using-dlarray-with-betarnd-randg#answer_1475101

  • Link

    Direct link to this answer

    https://www.mathworks.com/matlabcentral/answers/2130616-using-dlarray-with-betarnd-randg#answer_1475101

Edited: Matt J on 20 Jun 2024 at 23:00

Random number generation operations do not have derivatives in the standard sense. You will have to define some approximate derivative for yourself by implementing a backward() method.

2 Comments

Show NoneHide None

Matt J on 20 Jun 2024 at 23:17

Direct link to this comment

https://www.mathworks.com/matlabcentral/answers/2130616-using-dlarray-with-betarnd-randg#comment_3192086

Edited: Matt J about 17 hours ago

Open in MATLAB Online

You will have to define some approximate derivative for yourself by implementing a backward() method.

One candidate would be to reparametrize the beta distribution in terms of uniform random variables, U1 and U2, which you would save during forward propagation,

function [Z, U1, U2] = forward_pass(alpha, beta)

% Generate uniform random variables

U1 = rand();

U2 = rand();

% Generate Gamma(alpha, 1) and Gamma(beta, 1) using the inverse CDF (ppf)

X = gaminv(U1, alpha, 1);

Y = gaminv(U2, beta, 1);

% Combine to get Beta(alpha, beta)

Z = X / (X + Y);

end

During back propagation, your backward() method would differentiate non-stochastically with resepct to alpha and beta, using the saved U1 and U2 data as fixed and given values,

function [dZ_dalpha, dZ_dbeta] = backward_pass(alpha, beta, U1, U2, grad_gaminv)

% Differentiate gaminv with respect to the shape parameter alpha and beta

dX_dalpha = grad_gaminv(U1, alpha);

dY_dbeta = grad_gaminv(U2, beta);

% Compute partial derivatives of Z with respect to X and Y

X = gaminv(U1, alpha, 1);

Y = gaminv(U2, beta, 1);

dZ_dX = Y / (X + Y)^2;

dZ_dY = -X / (X + Y)^2;

% Use the chain rule to compute gradients with respect to alpha and beta

dZ_dalpha = dZ_dX * dX_dalpha;

dZ_dbeta = dZ_dY * dY_dbeta;

end

This assumes you have provided a function grad_gaminv() which can differentiate gaminv(), e.g.,

function grad = grad_gaminv(U, shape)

% Placeholder for the actual derivative computation of gaminv with respect to the shape parameter

% Here we use a numerical approximation for demonstration

delta = 1e-6;

grad = (gaminv(U, shape + delta, 1) - gaminv(U, shape, 1)) / delta;

end

DISCLAIMER: All code above was ChatGPT-generated.

Jack Hunt about 12 hours ago

Direct link to this comment

https://www.mathworks.com/matlabcentral/answers/2130616-using-dlarray-with-betarnd-randg#comment_3193201

  • Link

    Direct link to this comment

    https://www.mathworks.com/matlabcentral/answers/2130616-using-dlarray-with-betarnd-randg#comment_3193201

I see, so I do indeed need to use a closed form gradient. I had naively assumed that the autodiff engine would treat the stochastic (rng) quantities as non stochastic and basically do as you have described above.

Thank you for the answer. I shall work through the maths (re-derive the derivatives) and implement it the manual way. I have been spoiled by autodiff in the last decade or so; it’s been some time since I explicitly wrote a backward pass!

Sign in to comment.

More Answers (0)

Sign in to answer this question.

See Also

Categories

AI, Data Science, and StatisticsDeep Learning ToolboxImage Data Workflows

Find more on Image Data Workflows in Help Center and File Exchange

Tags

  • deep learning
  • statistics
  • matlab
  • neural networks
  • random number generator

Products

  • MATLAB
  • Deep Learning Toolbox
  • Statistics and Machine Learning Toolbox

Release

R2024a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

An Error Occurred

Unable to complete the action because of changes made to the page. Reload the page to see its updated state.


Using dlarray with betarnd/randg (5)

Select a Web Site

Choose a web site to get translated content where available and see local events and offers. Based on your location, we recommend that you select: .

You can also select a web site from the following list

Americas

  • América Latina (Español)
  • Canada (English)
  • United States (English)

Europe

  • Belgium (English)
  • Denmark (English)
  • Deutschland (Deutsch)
  • España (Español)
  • Finland (English)
  • France (Français)
  • Ireland (English)
  • Italia (Italiano)
  • Luxembourg (English)
  • Netherlands (English)
  • Norway (English)
  • Österreich (Deutsch)
  • Portugal (English)
  • Sweden (English)
  • Switzerland
    • Deutsch
    • English
    • Français
  • United Kingdom(English)

Asia Pacific

Contact your local office

Using dlarray with betarnd/randg (2024)
Top Articles
Latest Posts
Article information

Author: Pres. Lawanda Wiegand

Last Updated:

Views: 5738

Rating: 4 / 5 (71 voted)

Reviews: 86% of readers found this page helpful

Author information

Name: Pres. Lawanda Wiegand

Birthday: 1993-01-10

Address: Suite 391 6963 Ullrich Shore, Bellefort, WI 01350-7893

Phone: +6806610432415

Job: Dynamic Manufacturing Assistant

Hobby: amateur radio, Taekwondo, Wood carving, Parkour, Skateboarding, Running, Rafting

Introduction: My name is Pres. Lawanda Wiegand, I am a inquisitive, helpful, glamorous, cheerful, open, clever, innocent person who loves writing and wants to share my knowledge and understanding with you.