How to use stacked bar charts to draw multiple confidence intervals

How to use stacked bar charts to draw multiple confidence intervals as follows

My sample data is as follows: the point estimated coefficient is

coef = [-0.0186
0.0057
-0.0067
-0.0007
0
-0.0295
-0.0517
-0.0651
-0.0689
-0.0862
-0.0866
];

The lower bounds for the point estimated coefficient at 1%, 5%, 10%, 15% and 20% levels are

Lower_Bound = [
    -0.061944	-0.051528	-0.04632	-0.042792	-0.040104
-0.04203	-0.03056	-0.024825	-0.02094	-0.01798
-0.05314	-0.04198	-0.0364	-0.03262	-0.02974
-0.044302	-0.033824	-0.028585	-0.025036	-0.022332
0	0	0	0	0
-0.07723	-0.06576	-0.060025	-0.05614	-0.05318
-0.103042	-0.090704	-0.084535	-0.080356	-0.077172
-0.121602	-0.108024	-0.101235	-0.096636	-0.093132
-0.132368	-0.117116	-0.10949	-0.104324	-0.100388
-0.152506	-0.136572	-0.128605	-0.123208	-0.119096
-0.183092	-0.159904	-0.14831	-0.140456	-0.134472
];

The upper bounds for the point estimated coefficient at 1%, 5%, 10%, 15% and 20% levels are

Upper_Bound = [0.024744	0.014328	0.00912	0.005592	0.002904
0.05343	0.04196	0.036225	0.03234	0.02938
0.03974	0.02858	0.023	0.01922	0.01634
0.042902	0.032424	0.027185	0.023636	0.020932
0	0	0	0	0
0.01823	0.00676	0.001025	-0.00286	-0.00582
-0.000358	-0.012696	-0.018865	-0.023044	-0.026228
-0.008598	-0.022176	-0.028965	-0.033564	-0.037068
-0.005432	-0.020684	-0.02831	-0.033476	-0.037412
-0.019894	-0.035828	-0.043795	-0.049192	-0.053304
0.009892	-0.013296	-0.02489	-0.032744	-0.038728
];

[Hongyun]

@Umar @dpb

[Umar]

Hi @ Hongyun,

First, I will define the coefficients, lower bounds, and upper bounds as provided in your question.

% Coefficients

coef = [-0.0186; 0.0057; -0.0067; -0.0007; 0; -0.0295; -0.0517; -0.0651;     

-0.0689; -0.0862; -0.0866];

% Lower bounds for confidence intervals

   Lower_Bound = [

    -0.061944, -0.051528, -0.04632, -0.042792, -0.040104;
    -0.04203, -0.03056, -0.024825, -0.02094, -0.01798;
    -0.05314, -0.04198, -0.0364, -0.03262, -0.02974;
    -0.044302, -0.033824, -0.028585, -0.025036, -0.022332;
    0, 0, 0, 0, 0;
    -0.07723, -0.06576, -0.060025, -0.05614, -0.05318;
    -0.103042, -0.090704, -0.084535, -0.080356, -0.077172;
    -0.121602, -0.108024, -0.101235, -0.096636, -0.093132;
    -0.132368, -0.117116, -0.10949, -0.104324, -0.100388;
    -0.152506, -0.136572, -0.128605, -0.123208, -0.119096;
    -0.183092, -0.159904, -0.14831, -0.140456, -0.134472
   ];

% Upper bounds for confidence intervals

    Upper_Bound = [
    0.024744, 0.014328, 0.00912, 0.005592, 0.002904;
    0.05343, 0.04196, 0.036225, 0.03234, 0.02938;
    0.03974, 0.02858, 0.023, 0.01922, 0.01634;
    0.042902, 0.032424, 0.027185, 0.023636, 0.020932;
    0, 0, 0, 0, 0;
    0.01823, 0.00676, 0.001025, -0.00286, -0.00582;
    -0.000358, -0.012696, -0.018865, -0.023044, -0.026228;
    -0.008598, -0.022176, -0.028965, -0.033564, -0.037068;
    -0.005432, -0.020684, -0.02831, -0.033476, -0.037412;
    -0.019894, -0.035828, -0.043795, -0.049192, -0.053304;
    0.009892, -0.013296, -0.02489, -0.032744, -0.038728
  ];

Then calculate the heights of the bars to create the stacked bar chart. The heights for the lower bounds will be the difference between the coefficients and the lower bounds, while the heights for the upper bounds will be the difference between the upper bounds and the coefficients.

% Calculate the heights for the lower and upper bounds

lower_heights = coef - Lower_Bound;

upper_heights = Upper_Bound - coef;

Once you have the heights for the lower and upper bounds, you can create the stacked bar chart using the bar function.

% Create the stacked bar chart

figure;

bar([lower_heights, upper_heights], 'stacked');

% Set the x-axis labels

set(gca, 'XTickLabel', {'1%', '5%', '10%', '15%', '20%'});

xlabel('Confidence Levels');

ylabel('Coefficient Values');

 title('Stacked Bar Chart of Coefficients with Confidence Intervals');

legend({'Lower Bound', 'Upper Bound'}, 'Location', 'Best');

Finally, you can customize the chart to enhance its readability and presentation. This includes adding grid lines, adjusting colors, and ensuring that the legend is clear.

% Customize the appearance

grid on;

   % Custom colors for lower and upper bounds

   colormap([0.8 0.2 0.2; 0.2 0.8 0.2]);

Please see attached.

Feel free to adjust the aesthetics and parameters to better fit your specific needs and preferences. If you have any further questions, please let me know.

[Hongyun]

@Umar Hi bro, you did a good work. But the drawing results you provided are significantly different from the figures I want to get. As shown in the sample drawing, the estimated coefficient is located at the center, and the upper and lower bounds should be symmetrically distributed around the estimated coefficient. Just like this

Answer 1

[Umar]

Hi @ Hongyun,

So,your goal is to create a visually intuitive representation of these coefficients such that they are centered at zero on a stacked bar chart, with lower and upper bounds extending symmetrically around them. To achieve this, I need to adjust how I calculate and plot the heights of the bars in relation to the coefficients. Here’s an updated version of the code,

% Coefficients

coef = [-0.0186; 0.0057; -0.0067; -0.0007; 0; -0.0295; -0.0517; -0.0651;        
-0.0689; -0.0862; -0.0866];

% Lower bounds for confidence intervals

Lower_Bound = [
  -0.061944, -0.051528, -0.04632, -0.042792, -0.040104;
  -0.04203, -0.03056, -0.024825, -0.02094, -0.01798;
  -0.05314, -0.04198, -0.0364, -0.03262, -0.02974;
  -0.044302, -0.033824, -0.028585, -0.025036, -0.022332;
  0, 0, 0, 0, 0;
  -0.07723, -0.06576, -0.060025, -0.05614, -0.05318;
  -0.103042, -0.090704, -0.084535, -0.080356, -0.077172;
  -0.121602, -0.108024, -0.101235, -0.096636, -0.093132;
  -0.132368, -0.117116, -0.10949, -0.104324, -0.100388;
  -0.152506, -0.136572, -0.128605, -0.123208, -0.119096;
  -0.183092, -0.159904, -0.14831, -0.140456, -0.134472
];

% Upper bounds for confidence intervals

Upper_Bound = [
  0.024744, 0.014328, 0.00912, 0.005592, 0.002904;
  0.05343, 0.04196, 0.036225, 0.03234, 0.02938;
  0.03974, 0.02858, 0.023, 0.01922, 0.01634;
  0.042902, 0.032424, 0.027185, 0.023636, 0.020932;
  0, 0, 0, 0, 0;
  0.01823, 0.00676, 0.001025, -0.00286, -0.00582;
  -0.000358, -0.012696, -0.018865, -0.023044, -0.026228;
  -0.008598, -0.022176, -0.028965, -0.033564, -0.037068;
  -0.005432, -0.020684, -0.02831, -0.033476, -0.037412;
  -0.019894, -0.035828, -0.043795, -0.049192, -0.053304;
  0.009892, -0.013296, -0.02489, -0.032744, -0.038728
];

% Calculate heights for plotting

lower_heights = coef - Lower_Bound; % Height from coefficient to lower bound

upper_heights = Upper_Bound - coef; % Height from coefficient to upper 
bound

% Prepare data for stacked bar chart

data_to_plot = [Lower_Bound(:, 1), lower_heights, upper_heights];

% Create stacked bar chart

figure;

hBar = bar(data_to_plot, 'stacked');

% Set x-axis labels and other properties

set(gca, 'XTickLabel', {'1%', '5%', '10%', '15%', '20%'});

xlabel('Confidence Levels');

ylabel('Coefficient Values');

title('Stacked Bar Chart of Coefficients with Confidence Intervals');

legend({'Lower Bound', 'Coefficient', 'Upper Bound'}, 'Location', 'Best');

% Customize appearance

grid on;

colormap([0.8 0.2 0.2; 0 0 1; 0.2 0.8 0.2]); % Custom colors for each part

Please see attached.

So, after going through @dpb comments, I did concur with his comments. It was my fault not paying attention to comments.

% Coefficients

coef = [-0.0186, 0.0057, -0.0067, -0.0007, 0, -0.0295, -0.0517, 

-0.0651, ...
      -0.0689, -0.0862, -0.0866];

% Lower bounds for confidence intervals

Lower_Bound = [ -0.061944, -0.051528, -0.04632, -0.042792, 

-0.040104;
              -0.04203, -0.03056, -0.024825, -0.02094, -0.01798;
              -0.05314, -0.04198, -0.0364, -0.03262, -0.02974;
              -0.044302, -0.033824, -0.028585, -0.025036, -0.022332;
              0, 0, 0, 0, 0;
              -0.07723, -0.06576, -0.060025, -0.05614, -0.05318;
              -0.103042, -0.090704, -0.084535, -0.080356, -0.077172;
              -0.121602, -0.108024, -0.101235, -0.096636, -0.093132;
              -0.132368, -0.117116, -0.10949, -0.104324, -0.100388;
              -0.152506, -0.136572, -0.128605, -0.123208, -0.119096;
              -0.183092, -0.159904, -0.14831, -0.140456, -0.134472];

% Upper bounds for confidence intervals

Upper_Bound = [  0.024744, 0.014328, 0.00912, 0.005592, 0.002904;
               0.05343, 0.04196, 0.036225, 0.03234, 0.02938;
               0.03974, 0.02858, 0.023, 0.01922, 0.01634;
               0.042902, 0.032424, 0.027185, 0.023636, 0.020932;
               0, 0, 0, 0, 0;
               0.01823, 0.00676, 0.001025, -0.00286, -0.00582;
              -0.000358, -0.012696, -0.018865, -0.023044, -0.026228;
              -0.008598, -0.022176, -0.028965, -0.033564, -0.037068;
              -0.005432, -0.020684, -0.02831, -0.033476, -0.037412;
              -0.019894, -0.035828, -0.043795, -0.049192, -0.053304;
               0.009892, -0.013296, -0.02489, -0.032744, -0.038728];

% Reshape coef to match dimensions

coef_matrix = repmat(coef', 1, size(Lower_Bound, 2)); 

% Calculate differences for plotting

lower_heights = coef_matrix - Lower_Bound; % Height from coefficient 
to lower bound

upper_heights = Upper_Bound - coef_matrix; % Height from coefficient 
to upper bound

% Prepare data for stacked bar chart

data_to_plot = [Lower_Bound, lower_heights, upper_heights];

% Create stacked bar chart

figure;

hBar = bar(data_to_plot, 'stacked');

% Set x-axis labels and other properties

set(gca, 'XTickLabel', {'1%', '5%', '10%', '15%', '20%'});

xlabel('Confidence Levels');

ylabel('Coefficient Values');

title('Stacked Bar Chart of Coefficients with Confidence 
Intervals');

legend({'Lower Bound', 'Coefficient', 'Upper Bound'}, 'Location',   'Best');

% Customize appearance

grid on;

colormap([0.8 0.2 0.2; 0 0 1; 0.2 0.8 0.2]); % Custom colors for 
each part

Please see updated attached plot.

Hope this helps. Please let me know if you have any further questions.

[dpb]

@Umar, the first bar should range from -0.08054 -> -0.07018 for the lower tail...the bounds for the first coefficient unless I'm misreading the OP's definition of what his data are...and note again that your bounds aren't symmetric about the center as they should be; the thing about bar() adding the data is catching you out, methinks...

You'll note that yours has two sets of increasingly smaller sections in each bar; those are the upper and lower data negative and positive in order, just offset a little by you having added the coefficient value(*)

Look at the example in the doc that illustrates a simple case with negative numbers and you'll see what's happening...

(*) But, if I read the OP's explanation correctly, he's already got that in the bounds he's computed...he/she can clarify that to be sure, but otherwise it really doesn't make sense, the bounds otherwise should be a +/- single amount to be added/subracted from the coefficient.

[Hongyun]

Thanks bro.

[Hongyun]

@dpb @Umar I have encountered a new problem. In the sample diagram, the upper and lower bounds are symmetrically distributed around the estimated coefficients rather than 0. How can I obtain such figure using the test data?

[Umar]

Hi @ Hongyun,

It sounds like you are seeking a more visually intuitive representation of uncertainty in the estimates, which I understand is crucial in data analysis and presentation. So, to achieve the desired visualization where the upper and lower bounds are symmetrically distributed around the estimated coefficients, you will need to modify the existing code. Currently, the current structure of Lower_Bound and Upper_Bound arrays appears to not yield symmetry around the coefficients. So, I need to calculate new lower and upper bounds that are symmetric around each coefficient which can be done by determining a fixed margin (e.g., a percentage or a fixed value) from each coefficient. Here’s an updated version of code based on @dpb code that reflects these changes:

% Estimated coefficients

coef = [-0.0186 0.0057 -0.0067 -0.0007 0 -0.0295 -0.0517 -0.0651 -0.0689 
-0.0862 -0.0866];

% Define a margin for symmetry (you can adjust this value)

margin = 0.05; % Example margin; adjust based on your requirements

% Calculate symmetrical lower and upper bounds

Lower_Bound = coef - margin; % Lower bound

Upper_Bound = coef + margin; % Upper bound

% Combine lower and upper bounds into CI matrix

CI = [Lower_Bound' Upper_Bound']; % Transpose for proper dimensions

% Calculate differences for stacked bar chart

dCI = diff(CI, 1, 2);

% Create matrix for bar chart

M = [CI(:,1) dCI];

% Plotting

bar(M, 'stacked');

xlabel('Coefficients');

ylabel('Values');

title('Symmetrical Confidence Intervals Around Coefficients');

set(gca, 'XTickLabel', {'Coeff 1', 'Coeff 2', 'Coeff 3', 'Coeff 4', 'Coeff 5', ...
                       'Coeff 6', 'Coeff 7', 'Coeff 8', 'Coeff 9', 'Coeff 10'});

legend('Lower Bound', 'Upper Bound');

grid on;

Please bear in mind that the choice of margin is critical as it directly affects the visual representation of uncertainty. You may want to base this on statistical considerations relevant to your data (e.g., standard errors or confidence levels). Hope this helps. Please let me know if you have any further questions.

[Hongyun]

@Umar @dpb I ran the code you provided, but the result doesn't seem to be correct. I think I need to further search for the right solution.

Let me provide a vector of standard errors for the estimates,

standard_error = [0.0168000000000000
0.0185000000000000
0.0180000000000000
0.0169000000000000
0
0.0185000000000000
0.0199000000000000
0.0219000000000000
0.0246000000000000
0.0257000000000000
0.0374000000000000];

Then, If we want to obtain an upper bound at the 1% level, we can calculate it by

Simiarily, if we want to obtain an lower bound at the 1% level, we can calculate it by

In fact, the upper and lower bounds I provided above were calculated using this method, so the upper and lower bounds should be symmetrically distributed around the estimated coefficients.

[Hongyun]

@Umar@dpb Noted that, the five columns of upper and lower bounds correspond to confidence intervals for estimated coefficients at 99%, 95%, 90%, 85%, and 80% levels, respectively.

[Umar]

Hi @Hongyun,

After reading your comments, I started with the vector of standard errors you've provided.

% Given standard errors and coefficients

standard_error = [0.0168; 0.0185; 0.0180; 0.0169; 0; 0.0185;   0.0199; 0.0219;0.0246; 0.0257; 0.0374];

For each confidence level, I determined the critical value from the Z-distribution (normal distribution). For example: For a 99% confidence level, the critical value (Z) is approximately 2.576.

For a 95% confidence level, Z ≈ 1.960. For a 90% confidence level, Z ≈ 1.645. For an 85% confidence level, Z ≈ 1.440. For an 80% confidence level, Z ≈ 1.282.

Then, use these critical values to calculate the upper and lower bounds for each coefficient.

   coef = [-0.0186, 0.0057, -0.0067, -0.0007, 0, -0.0295, -0.0517,   
   -0.0651, -0.0689, -0.0862, -0.0866];

% Define critical values for each confidence level

critical_values = [2.576; 1.960; 1.645; 1.440; 1.282];

% Initialize matrices for lower and upper bounds

   Lower_Bound = zeros(length(coef), length(critical_values));

   Upper_Bound = zeros(length(coef), length(critical_values));

   % Calculate bounds

    for i = 1:length(critical_values)

      Lower_Bound(:, i) = coef' - critical_values(i) * 
      standard_error;

      Upper_Bound(:, i) = coef' + critical_values(i) * 
      standard_error;

end

Then, concatenate lower and upper bounds into a single matrix for plotting.

   % Combine lower and upper bounds into a single matrix

    CI = [Lower_Bound Upper_Bound];

    % Calculate differences for stacked bar chart

    dCI = diff(CI, 1, 2);

    M = [CI(:,1) dCI];    % Prepare data for bar chart

Finally, created a stacked bar chart to visualize these bounds.

      % Create a stacked bar chart

       bar(M, 'stacked');

       xlabel('Coefficients');

        ylabel('Values');

       title('Symmetrical Confidence Intervals Around 
       Coefficients');

       % Set x-tick labels

        set(gca, 'XTickLabel', {'Coeff 1', 'Coeff 2', 'Coeff 3',                 'Coeff 4', 'Coeff 5', ... 'Coeff 6', 'Coeff 7', 'Coeff 8',         'Coeff 9', 'Coeff 10'});

      % Define legend entries correctly

        legend('Lower Bound (99%)', 'Upper Bound (99%)', 
        'Location', 'Best');

        grid on;

The selection of confidence levels is crucial as it dictates how much uncertainty you are willing to accept in your estimates. Also, the resulting plot will provide a clearer visual representation of how each coefficient estimate varies within its respective confidence interval, aiding in better decision-making based on statistical evidence. Please make sure that your data meets the assumptions required for calculating standard errors accurately—this includes normality and independence of residuals in regression analysis. Hope this should help resolve your problem. Feel free to reach out if you have further questions or require additional clarification!

[Hongyun]

@Umar Bro, thanks for you great work again! When I ran your code in MATLAB and got the following figure

How should I adjust the color to clearly observe which level of significance the estimated coefficient has passed？ Just like this sample figure

[Hongyun]

@Umar @dpb As shown in sample figure (obtained by the Stata Software), we can clearly observe that the coefficient at period t pass the sigificance level of 5% (since the 95% confidence interval do not include zero), but it did not pass the sigificance level of 1% (since the 99% confidence interval includes zero). However, I am unable to clearly achieve this goal in the MATLAB code you provided.

[Umar]

Hi @Hongyun,

Give me some time to think about it. In the meantime, explain in terms that helps clarify what exactly is missing in the code compared to SATA software from recent code provided like what are the missing requirements, is it missing a line across the stacked bars, etc

[dpb]

I fail to see how, other than the graphics, the sample figure has anything to do with the actual data set you provided.

I noted that one would need to fiddle with the color map to be able to get sufficient gradation between the bar sections with a unicolor scheme; that's simply adjusting the values in the color map array somewhat.

However, with the narrowness of the bands in your dataset, I suspect it will quite difficult to ascertain that clearly, regardless of the colormap; the bands are so narrow as to make distinguishing one from another quite difficult.

Also note I did not adjust the height of the bars by the manitude of the coefficient thinking from your explanation you had already computed the bands based on the coefficient values and error. If that is not so, then the bottom data point in my code needs to have the magnitude of the coefficient for each bar adjusted by the value of the specific coefficient -- just add to the lower bound value.

If you want clear visualization of a given confidence level such as 95%, just for checking that are getting the expected result, use a binary colormap of gray/black between the two ranges and it will be apparent.

But, the basic problem is twofold --

1. Finding a good colormap to be able to distinguish the five ranges, and
2. The data bounds are so small as the size of the tail subsections is virtually indistinguishable regardless the color.

[dpb]

"@Umar Bro, thanks for you great work again! When I ran your code in MATLAB and got the following figure..."

The problem here is the same one @Umar has run into before -- bar() does NOT plot the actual values for a stacked plot, but adds the values cumulatively, and in the order they are in each row. His calculation of the critical points is from the tail percentage to the lower in both directions; this accounts for the upper tail elements being at the bottom of the section above zero instead of at the top and symmetrical to the bottom about the (should be) middle. To do this, as my code shows, need to reverse the upper z-values order.

The use of the absolute values distorts what is intended as well, the bars are not symmetric around either zero nor the location of the coefficient value (the latter being the intent); this is because of the aforementioned behavior of bar() with stacked data -- it plots the cumsum of the values in order, but all negative values add below the zero-axis and all positive values above. To end up with bar sections at specific points with bar(), you must plot the differences between subsequent sections respecting both the sign of the differences and the order (and, the sign is dependent upon the order being right; ergo the minus signs in my code to account for the order given in the original example;

My code has the basics to do this other than it doesn't adjust the midpoint; I presumed it had already been done, but if not that needs to be added.

In general, one might find this a much easier problem to solve in MATLAB with patch instead of bar() -- then you can, in fact, use absolute coordinates to draw the subsections where desired without the extra machinations to try to beat bar() into submission.

[dpb]

y=[-3 -2 1 6;nan(1,4)];  % bar is broke; won't stack unless at least two rows...
bar(y,'stacked')        

NOTA BENE the above simple example -- the two negative values are below zero as expected in the sequence from the origin they are in the input vector...but they add to -5, not the positions of the two values; that's why when you compute a percentile and try to plot it at that position, you have to pass bar() the difference in the values from the first and in the order of middle-to-negative largest -> middle-to-positive largest.

[Hongyun]

@dpb I think perhaps the root of the problem lies not in the data, but in the code you provided. Based on the same data, we can use a strip chart to draw the corresponding confidence intervals at any significance level. For example, the following figure plots the results at the 5% significance level

As shown in this figure, we can clearly observed that the estimated coefficient at passed the 5% significance level (since the upper and lower bounds does not include zero). However, one obvious flaw of this strip chart is that we cannot determine the significance of the coefficients at multiple levels. That is, we don't know if the coefficient is significant at any level below 5% (e.g., 1%, 2%, 3%, 4%, etc). In contrast, stacking bar charts can help us achieve this goal, which is also why I asked this question.

[Hongyun]

@dpb Of course, we can draw multiple significance level strip plots based on upper and lower bounds, but this is not the best solution to answer my question. I need to use MATLAB software to draw figures similar to Stata software. Just like this one

[Hongyun]

@dpb From the code implementation results you provided, it appears to be very close to the actual results, but there still seem to be some minor flaws. As in my conversation with @Umar, I have provided estimated coefficients and corresponding standard errors. If you feel that there are issues with the upper and lower bounds I provided, you can generate your own upper and lower bounds at any level of significance.

% Given standard errors and coefficients

standard_error = [0.0168; 0.0185; 0.0180; 0.0169; 0; 0.0185; 0.0199; 0.0219;0.0246; 0.0257; 0.0374];

coef = [-0.0186, 0.0057, -0.0067, -0.0007, 0, -0.0295, -0.0517, -0.0651, -0.0689, -0.0862, -0.0866];

[dpb]

"still seem to be some minor flaws. "

On further examination of the behavior of bar(), I don't think it can be coerced into doing precisely what is wanted without even a great deal more machinations in computing the various differences above and below zero to match.

I think the only hope would be to use an offset to make all values single-signed (positive would be easier) and draw the bar with those values and then adjust the y-axis tick values to match the actual data range by subtracting the offset number.

Unfortunately, I have other commitments at the moment, but I think I will reiterate my recommendation to use patch here instead; the MATLAB bar() function just doesn't have the flexibility you need to do this easily, although the above workaround might not be too bad; with that adjustment @Umar's code may work more as expected; it would solve his problem of the reflected sections about the origin. Maybe he'll have a few moments to make the adjustments to move the values up all to be positive in his code and then fixup the axes labels...or, you could try making those adjustments and post back where you might get stuck.

My idea is sorta' on the right track, but the way bar() reflects stuff about the orign means it simply cannot function as desired; when you add a difference to the negative value that causes the next to be positive, bar() will place that above the axis. not straddling the axis from the top of the previous section.

[Umar]

Hi @Hongyun ,

To address your concerns in posted comments and save time, I implemented and modified existing code. So, definitions of the standard errors, coefficients, and their respective confidence intervals (lower and upper bounds) used to create the visual representation will stay the same based on your comments as shown below.

% Given standard errors 

standard_error = [0.0168; 0.0185; 0.0180; 0.0169; 0; 0.0185; 0.0199; 0.0219; 
0.0246; 0.0257; 0.0374];

% Coefficients and Confidence Intervals

coef = [-0.0186 0.0057 -0.0067 -0.0007 0 -0.0295 -0.0517 -0.0651 -0.0689 -
-0.0862 -0.0866];

Lower_Bound = [ -0.061944 -0.051528 -0.04632 -0.042792 -0.040104
              -0.04203  -0.03056  -0.024825 -0.02094  -0.01798
              -0.05314  -0.04198  -0.0364   -0.03262  -0.02974
              -0.044302 -0.033824 -0.028585 -0.025036 -0.022332
               0        0         0        0        0
              -0.07723  -0.06576  -0.060025 -0.05614  -0.05318
              -0.103042 -0.090704 -0.084535 -0.080356 -0.077172
              -0.121602 -0.108024 -0.101235 -0.096636 -0.093132
              -0.132368 -0.117116 -0.10949  -0.104324 -0.100388
              -0.152506 -0.136572 -0.128605 -0.123208 -0.119096
              -0.183092 -0.159904 -0.14831  -0.140456 -0.134472];

Upper_Bound = [  0.024744  0.014328  0.00912   0.005592  0.002904
               0.05343   0.04196   0.036225  0.03234   0.02938
               0.03974   0.02858   0.023     0.01922   0.01634
               0.042902  0.032424  0.027185  0.023636  0.020932
               0         0         0         0         0
               0.01823   0.00676   0.001025  -0.00286  -0.00582
              -0.000358  -0.012696 -0.018865 -0.023044 -0.026228
              -0.008598  -0.022176 -0.028965 -0.033564 -0.037068
              -0.005432  -0.020684 -0.02831  -0.033476 -0.037412
              -0.019894  -0.035828 -0.043795 -0.049192 -0.053304
               0.009892  -0.013296 -0.02489  -0.032744 -0.038728];

Next, a matrix created below that combines the lower and upper bounds of the confidence intervals which is used to generate the stacked bar plot is based on code provided by @dpb

% Create Confidence Interval Matrix

CI = [Lower_Bound Upper_Bound];

dCI = diff(CI, 1, 2); % Calculate the difference for the height of the bars

M = [CI(:, 1) dCI];   % Combine lower bounds and differences

Then, the stacked bar plot created to visualize the confidence intervals, and overlay the coefficients and standard errors is again based on @dpb’s code below.

% Create Stacked Bar Plot

figure;

bar(M, 'stacked'); % Create the stacked bar plot

hold on;

% Plot Coefficients

x = 1:length(coef);

plot(x, coef, 'k-o', 'MarkerFaceColor', 'k', 'LineWidth', 2);

% Plot Standard Errors

errorbar(x, coef, standard_error, 'k.', 'MarkerSize', 10, 'LineWidth', 1.5);

To enhance the visualization, I color coded the coefficients based on their significance levels.

% Color Coding for Significance Levels

for i = 1:length(coef)

    if coef(i) < 0

        color = 'r'; % Red for negative coefficients

    elseif coef(i) > 0

        color = 'g'; % Green for positive coefficients

    else

        color = 'b'; % Blue for zero coefficients

    end

    plot(x(i), coef(i), 'd', 'Color', color, 'MarkerFaceColor', color, 'MarkerSize', 8);

end

Finally, I added labels, a title, and a legend to the plot to ensure clarity and comprehensibility.

% Labels and Title

xlabel('Coefficient Index');

ylabel('Value');

title('Coefficients with Confidence Intervals, Standard Errors, and Significance   
Levels');

legend('Lower Bound', 'Upper Bound', 'Coefficients', 'Standard Errors', '
 ‘Location', 'Best');

hold off;

So, the provided modified code of @dpb visualizes the coefficients along with their confidence intervals and standard errors. The use of color coding enhances the interpretability of the coefficients, allowing for quick identification of their significance.

Please see attached plot.

Hope this helps. Please let me know if you have any further questions.

My analysis of comments regarding stacked plot

I do concur with @dpb comments about, “In general, one might find this a much easier problem to solve in MATLAB with patch instead of bar() -- then you can, in fact, use absolute coordinates to draw the subsections where desired without the extra machinations to try to beat bar() into submission.”

As I mentioned in my comments, I needed to do a little research on side which was beneficial because it did help me understand a little bit more about patch function. Also, it helped me understand concept about negative values included in stacked bar when I came across, this link with header shown below.

Stacked bar graph with negative BaseValue but "positive" height

https://www.mathworks.com/matlabcentral/answers/2010977-stacked-bar-graph-with-negative-basevalue-but-positive-height

My interpretation of comments between @Voss and OP: So, basically in nutshell, this is what I interpreted from their conversation that stacked bar graphs are designed to show how different parts contribute to a whole. Each segment of the bar represents a category (in this case, costs and revenues), and their cumulative height reflects the total value. However, when dealing with dynamic data where values might change frequently, as you’ve experienced, using standard bar functions can lead to unexpected results or require complex adjustments.

Why Use patch instead of bar

_ Control Over Appearance_

The patch function allows for greater customization of each segment's appearance. You can specify colors, transparency, and edge properties more easily than with bar. For example, in link above, it is shown using different color arrays for costs and revenues with `patch`, which visually distinguishes these components more effectively.

_ Single Axes Requirement_

When using patch, you can plot multiple datasets on a single axes object without needing to create separate axes for each dataset. This reduces clutter in your visualization and simplifies the plotting process. Again, in link above, it successfully combined costs and revenues into one cohesive plot rather than needing multiple calls to separate bar plots.

If you noted @Voss discussion with Leon Berger, changing parameters like rev_distr or costs_distr can be more intuitive with patch. It does allow you to redraw only specific segments without recalibrating the entire graph.This flexibility is especially valuable in scenarios where data is frequently updated or modified such in your case. Also, stacked bars can sometimes lead to misleading interpretations if not constructed correctly (e.g., if base values are misaligned), so using patch directly defines the coordinates for each section, minimizing these risks. Now, the most important thing to notice that when using patch, you should maintain the integrity of your data representation. Double-check that the cumulative totals are accurately reflected in your visual output. While using patch provides flexibility, it may also require more manual input in terms of data handling. If performance becomes an issue with large datasets, consider optimizing your data processing before visualization.

Now to address my comments regarding what @dpb mentioned, “The problem here is the same one @Umar has run into before in matlab -- bar() does NOT plot the actual values for a stacked plot, but adds the values cumulatively, and in the order they are in each row. His calculation of the critical points is from the tail percentage to the lower in both directions; this accounts for the upper tail elements being at the bottom of the section above zero instead of at the top and symmetrical to the bottom about the (should be) middle,”

I was trying to find workaround this issue, so I did some research if Stata faced this issue and found a workaround it, so I could resolve this issue. So, after some digging, I found out seems that I was not the only one having this issue in Matlab, it also occurred in Stata software which was kept mentioning by OP (@)Hongyun ). Please click the link below and go to section where Nick Cox (Active Stata User) made the following comments: “29 Jun 2023, 03:13, This may help in the absence of a data example. The idea is to stack bar segments below 0 if negative and above 0 otherwise. That obliges keeping track of the sums of positives and negatives so far.”

https://www.statalist.org/forums/forum/general-stata-discussion/general/1587510-is-it-possible-to-overlay-a-stacked-bar-plot-and-a-line-plot

Now, if you compare the above observations mentioned above to the comments from Nick Cox, it appears that both of us were addressing similar challenges related to plotting stacked bar segments based on positive and negative values. However, Nick Cox suggested a workaround by stacking bar segments below 0 for negative values and above 0 for positive values while keeping track of the sums of positives and negatives to maintain consistency in the representation of data.

In my opinion, I did not find the idea of stacked bar being so useful especially about using bar function to plot stacked bar charts to draw multiple confidence intervals but going through bunch of manipulation and calculations beforehand to get results and when you keep adding data, patch function is useful in this case. Also, another caveat regarding to stackedbar was about dataset being involved with negative values and Stata software having issues with this as well which is clearly evident from user who used their software and struggled as well.

[dpb]

@Umar - "the provided modified code of @dpb visualizes the coefficients along with their confidence intervals and standard errors."

But you forgot to reverse the order of the upper bar segments so the tails are reflected about the origin; again the upper tail percentages are plotted at the base of the positive axis for each bar...but, while that can make it look nice and symmetric one simply cannot make bar() put the bar sections that should cross the origin where they should go to stack them to cross the origin; it will put an entire bar above/below the line and fill the cumulative difference. You note that the middle confidence inverval that should be symmetric about the origin is split into two at the origin -- bar() will never draw a single segment of a stacked bar crossing the origin.

y=[-1.96 -1.645 1.645 1.96;nan(1,4)];
bar(y,'stacked')

Drawing the z values shows the problem -- without using the deltas, the total is not proportional to the two percentiles but the cumulative sum of the values. That's why the bar segments in your plots were so much more obvious than in mine; the difference between z values is much less than the z value itself.

And, instead of having the desired effect of there being a bar between 1.96 and 1.645 and another between -1.645 and +1.645, bar() draws the two from the origin in the signed direction and then adds the other on top/bottom.

All in all, use patch(). I also haven't searched FEX or web to see if anybody has already built a function to replicate the Stata graphic...

[Umar]

Hi @dpb,

First, I do appreciate exchanging thoughts with each other about visual representation which was interesting. In my opinion, my MATLAB code implementation using stacked bar plots serves its purpose well for visualizing coefficients along with confidence intervals and standard errors, regarding visualizing data using patch(), which addresses the challenges encountered with negative values in stacked bar plots, here is updated code,

This part of the code remains the same

% Given standard errors 
standard_error = [0.0168; 0.0185; 0.0180; 0.0169; 0; 0.0185;                         0.0199; 0.0219; 0.0246; 0.0257; 0.0374];

% Coefficients and Confidence Intervals
coef = [-0.0186; 0.0057; -0.0067; -0.0007; 0; -0.0295; -0.0517;                           -0.0651;  -0.0689; -0.0862; -0.0866];

% Lower and Upper Bounds
Lower_Bound = [ -0.061944; -0.051528; -0.04632; -0.042792;                                           -0.040104; -0.04203;  -0.03056;  -0.024825;                                   -0.02094;  -0.01798; -0.05314];

Upper_Bound = [  0.024744;  0.014328;  0.00912;   0.005592;                                                 0.002904;0.05343; 0.04196; 0.036225;  0.03234;      
                0.02938;0.03974];

Then, create x-coordinates for the coefficients and prepare the y-coordinates for the lower and upper bounds.

% Prepare data for patch

x = 1:length(coef);

y_lower = Lower_Bound;

y_upper = Upper_Bound;

Initialize a figure and hold it for multiple plots.

% Create figure

figure;

hold on;

Using a loop to plot the confidence intervals using the patch function for both lower and upper bounds.

% Plot each section using patch
for i = 1:length(coef)
  % Define x-coordinates for patch
  x_patch = [x(i) x(i) x(i) x(i)];

    % Define y-coordinates for lower bound (negative)
    y_patch_lower = [y_lower(i), y_lower(i), 0, 0];

    % Define y-coordinates for upper bound (positive)
    y_patch_upper = [y_upper(i), y_upper(i), 0, 0];

    % Draw lower bound patch (negative)
    patch(x_patch(1:2), y_patch_lower(1:2), 'r', 'FaceAlpha', 0.5);

    % Draw upper bound patch (positive)
    patch(x_patch(3:4), y_patch_upper(3:4), 'g', 'FaceAlpha', 0.5);
  end

Plotting the coefficients and overlay the standard errors using the errorbar function.

% Overlay coefficients plot(x, coef, 'k-o', 'MarkerFaceColor', 'k', 'LineWidth', 2);

% Overlay standard errors
errorbar(x, coef, standard_error, 'k.', 'MarkerSize', 10,   
 ‘LineWidth', 1.5);

Enhancing the plot with appropriate labels and a title for clarity and releasing the hold on the figure to finalize the plot.

   hold off;

Please see attached.

Hope, this will suffice for now.

[dpb]

" In my opinion, MATLAB code implementation using stacked bar plots serves its purpose well for visualizing coefficients along with confidence intervals and standard errors,"

It's pretty, yes, but sadly it's simply not correct in any incarnation so far...

[Umar]

Hi @dpb,

Could you point out what modifications needs to be added to code below regarding, “ sadly it's simply not correct in any incarnation so far.” since you mentioned about using patch which I already did.

% Given standard errors standard_error = [0.0168; 0.0185; 0.0180; 0.0169; 0; 0.0185; 0.0199; 0.0219; 0.0246; 0.0257; 0.0374]; % Coefficients and Confidence Intervals coef = [-0.0186 0.0057 -0.0067 -0.0007 0 -0.0295 -0.0517 -0.0651 -0.0689 - -0.0862 -0.0866]; Lower_Bound = [ -0.061944 -0.051528 -0.04632 -0.042792 -0.040104 -0.04203 -0.03056 -0.024825 -0.02094 -0.01798 -0.05314 -0.04198 -0.0364 -0.03262 -0.02974 -0.044302 -0.033824 -0.028585 -0.025036 -0.022332 0 0 0 0 0 -0.07723 -0.06576 -0.060025 -0.05614 -0.05318 -0.103042 -0.090704 -0.084535 -0.080356 -0.077172 -0.121602 -0.108024 -0.101235 -0.096636 -0.093132 -0.132368 -0.117116 -0.10949 -0.104324 -0.100388 -0.152506 -0.136572 -0.128605 -0.123208 -0.119096 -0.183092 -0.159904 -0.14831 -0.140456 -0.134472]; Upper_Bound = [ 0.024744 0.014328 0.00912 0.005592 0.002904 0.05343 0.04196 0.036225 0.03234 0.02938 0.03974 0.02858 0.023 0.01922 0.01634 0.042902 0.032424 0.027185 0.023636 0.020932 0 0 0 0 0 0.01823 0.00676 0.001025 -0.00286 -0.00582 -0.000358 -0.012696 -0.018865 -0.023044 -0.026228 -0.008598 -0.022176 -0.028965 -0.033564 -0.037068 -0.005432 -0.020684 -0.02831 -0.033476 -0.037412 -0.019894 -0.035828 -0.043795 -0.049192 -0.053304 0.009892 -0.013296 -0.02489 -0.032744 -0.038728]; Next, a matrix created below that combines the lower and upper bounds of the confidence intervals which is used to generate the stacked bar plot is based on code provided by @dpb

% Create Confidence Interval Matrix CI = [Lower_Bound Upper_Bound]; dCI = diff(CI, 1, 2); % Calculate the difference for the height of the bars M = [CI(:, 1) dCI]; % Combine lower bounds and differences Then, the stacked bar plot created to visualize the confidence intervals, and overlay the coefficients and standard errors is again based on @dpb’s code below.

% Create Stacked Bar Plot figure; bar(M, 'stacked'); % Create the stacked bar plot hold on; % Plot Coefficients x = 1:length(coef); plot(x, coef, 'k-o', 'MarkerFaceColor', 'k', 'LineWidth', 2); % Plot Standard Errors errorbar(x, coef, standard_error, 'k.', 'MarkerSize', 10, 'LineWidth', 1.5); To enhance the visualization, I color coded the coefficients based on their significance levels.

% Color Coding for Significance Levels for i = 1:length(coef) if coef(i) < 0 color = 'r'; % Red for negative coefficients elseif coef(i) > 0 color = 'g'; % Green for positive coefficients else color = 'b'; % Blue for zero coefficients end plot(x(i), coef(i), 'd', 'Color', color, 'MarkerFaceColor', color, 'MarkerSize', 8); end Finally, I added labels, a title, and a legend to the plot to ensure clarity and comprehensibility.

% Labels and Title xlabel('Coefficient Index'); ylabel('Value'); title('Coefficients with Confidence Intervals, Standard Errors, and Significance Levels'); legend('Lower Bound', 'Upper Bound', 'Coefficients', 'Standard Errors', ' ‘Location', 'Best'); hold off;

[Hongyun]

@Umar I have tried the ideas you provided, and perhaps I have obtained the answer I wanted. Thank you very much for your help.

@dpb I have also carefully reviewed your response, and although you have provided answers that are very close to the actual results, after verification, your code is still incorrect. Therefore, I have to say that @Umardid better than you on this issue, but you also provided some constructive suggestions, so I also want to thank you.

[dpb]

@Hongyun - yes, I agreed earlier that despite my earlier belief the initial code was correct, later more detailed examination showed it also had some issues despite superfically looking reasonable. I simply did/do not have the time available at the moment to dig into it more throughly.

@Umar At the time I wrote that, no results that @Hongyun had shown (and I had seen, anyway) didn't have at least one of the issues noted...the last figure above he posted appears reasonable...I didn't have the time to try to format the code to be able to see just what/how it was done.

[Hongyun]

@Umar @dpb Thank you for your help. In the future, I will pose more drawing problems to challenge you, and I believe we can all improve our programming skills from them.

[Hongyun]

@Umar @dpb In the six questions I have raised in the past, you have been actively helping me solve them. I am truly grateful and hope that we can become good partners in the future.

[Umar]

Hi @Hongyun,

Thank you for your kind words. I am pleased to hear that my assistance has been helpful in addressing the questions you raised. It is always my goal to provide support and facilitate effective communication. I share your hope for a fruitful partnership moving forward.

Answer 2

[dpb]

coef = [-0.0186 0.0057 -0.0067 -0.0007 0 -0.0295 -0.0517 -0.0651 -0.0689 -0.0862 -0.0866];
Lower_Bound = [ -0.061944	-0.051528	-0.04632	-0.042792	-0.040104
                -0.04203	-0.03056	-0.024825	-0.02094	-0.01798
                -0.05314	-0.04198	-0.0364	    -0.03262	-0.02974
                -0.044302	-0.033824	-0.028585	-0.025036	-0.022332
                 0	         0	         0	         0	         0
                -0.07723	-0.06576	-0.060025	-0.05614	-0.05318
                -0.103042	-0.090704	-0.084535	-0.080356	-0.077172
                -0.121602	-0.108024	-0.101235	-0.096636	-0.093132
                -0.132368	-0.117116	-0.10949	-0.104324	-0.100388
                -0.152506	-0.136572	-0.128605	-0.123208	-0.119096
                -0.183092	-0.159904	-0.14831	-0.140456	-0.134472];
Upper_Bound = [  0.024744	 0.014328	 0.00912	 0.005592	 0.002904
                 0.05343     0.04196	 0.036225	 0.03234	 0.02938
                 0.03974     0.02858	 0.023	     0.01922	 0.01634
                 0.042902	 0.032424	 0.027185	 0.023636	 0.020932
                 0	         0	         0	         0	         0
                 0.01823	 0.00676	 0.001025	-0.00286	-0.00582
                -0.000358	-0.012696	-0.018865	-0.023044	-0.026228
                -0.008598	-0.022176	-0.028965	-0.033564	-0.037068
                -0.005432	-0.020684	-0.02831	-0.033476	-0.037412
                -0.019894	-0.035828	-0.043795	-0.049192	-0.053304
                 0.009892	-0.013296	-0.02489	-0.032744	-0.038728];
CI=[Lower_Bound Upper_Bound];
dCI=diff(CI,1,2);
M=[CI(:,1) dCI];
bar(M,'stacked')

The problem with @Umar's solution is that bar() adds up all the data by row so that all the negative values in the original add below the line and all the positives above which all end up negating each other so the total is about zero.

To fool Mother Bar(), you have to start at the bottom and let it add the differences; then you'll get the bars centered about zero as the example. The above, however, also shows that you have to order the percentages from center to end in both directions facing the signs; you'll see the tails are both at the bottom in the above with the given orientation...we'll leave that as "exercise for Student"...

ADDENDUM:

OK, had a few minutes...try this for starters --

dCL=-diff(Lower_Bound,1,2);                 % difference between pcts, signed lower bounds
dCU=-diff(Upper_Bound,1,2);                 % ditto for upper tail percentages
dZ=Upper_Bound(:,1)-Lower_Bound(:,end);     % the difference from last lower to first upper %age
M=[Lower_Bound(:,1) dCL dZ dCU];            % combine differences starting with bottom tail absolute
figure
hB=bar(M,'stacked','FaceColor','flat');     % plot, prepare to change colors by colormap
m=colormap(bone(7)); m=m(2:end-1,:);        % pick over range of given colormap excluding black/white
N=numel(hB)/2;                              % how many each half of percentages
for i=1:N                                   % iterate over that number
  j=N+i;                                    % the remaining (upper tail)
  hB(i).CData=m(i,:);                       % set lower tail bars
  hB(j).CData=m(i,:);                       % and upper tail bars to match symmetrically
end

NOTA BENE: The negation of the differences -- as given in order from 0-100%, the change in magnitude is positive for the LH/lower tail and negative for the upper/RH. Hence besides the order of both tails being at the bottom of the bar sections already noted, the tail percentages are plotted on the wrong ends; negative values belonging to the RH tail are below the axis. The negation of the calculation of the differences fixes this so they will be on the correct end.

Also, since bar() works only with the cumulative sum of values above and below the zero axis value, to move up the axis from the last lower percentage one must calculate and add the difference between the last given LH point and the first RH point (dZ in above code), otherwise the tail percentages for the upper tail would just begin at the origin adding only the tail percentage differences.

Will take some futzing with colormap to get better gradation, but above shows the technique to control the individual bar colors -- each section of each bar is a separate bar() handle.

[Umar]

Thanks for your help, @dpb, I really appreciate it.

[Hongyun]

@dpb @Umar Thank you for your answer. I really appreciate it! You are the best partners and also very top programming mentors for me.

[Umar]

Hi @ Hongyun,

Thank you for your kind words. I truly appreciate your feedback and am delighted to hear that you find our partnership valuable. It is a pleasure to support you on your programming journey, and I look forward to continuing our collaboration. If there’s anything specific you would like to discuss or any further assistance you need, please do not hesitate to reach out.