Python Graph & Visualization

数据科学

matplotlib

基础语法与概念

plotly

matplotlib VS plotly

1. matplotlib can use custom color and line style
2. plotly is more easily to quickly built up.

基础语法与概念

线性颜色柱的选择

https://plotly.com/python/builtin-colorscales/

same in matplotlib

plt.show 转发图像到本地

使用Dash: A web application framework for your data., 默认部署在localhost:8050端口

本地机器打通ssh隧道

ssh -L 8050:127.0.0.1:8050 -vN -f -l shaojiemike 202.38.72.23

科研画图

In scientific research plotting, it’s always bar charts instead of line charts.

Possible reasons:

1. The visual footprint of point lines is relatively small compared to bar
2. Line charts are aggregated together when there is a meaningful comparison of data along the y-axis.

global setting

font things, Attention, global settings have higher priority to get work

matplotlib.rcParams.update({
    "pgf.texsystem": "pdflatex",
    'font.family': 'serif',
    # 'text.usetex': True, # comment to support bold font in legend, and font will be bolder
    # 'pgf.rcfonts': False,
})

layout

1. picture size

2. figure size (adjust x,y tick distance)

# mpl
fig.set_size_inches(w= 0.5 * x_count * (group_count+1.5), h=5.75 * 0.8) #(8, 6.5)

# plotly
fig.update_layout(height=350, width=50 * graph_data.size(),
                  margin=dict(b=10, t=10, l=20, r=5),
                  bargap=0.2 # x tick distance, 1 is the normalize-distance of adjacent bars
                  )

3. relative position

# mpl: Adjust the left margin to make room for the legend, left & right chart vertical line position from [0,1]
plt.subplots_adjust(left=0.1, right=0.8)

set x axis

# mpl:
x = np.arange(len(graph_data.x))  # the label locations, [0, 1, 2, 3, 4, 5, 6, 7]
# set the bar move to arg1 with name arg2
ax.set_xticks(x + (group_count-1)*0.5*width, graph_data.x)
plt.xticks(fontsize=graph_data.fontsize+4)
# Adjust x-axis limits to narrow the gap
plt.xlim(-(0.5+gap_count)*width, 
        x_count - 1 + (group_count-1)*width + (0.5+gap_count)*width)
    

# plotly

set y axis

1. vertical grid line
2. dick size
3. and range size

# mpl
ax.set_ylabel(yaxis_title, 
              fontsize=graph_data.fontsize,
              fontweight='bold')
plt.grid(True, which='major',axis='y', zorder=-1.0) # line and bar seems need to set zorder=10 to cover it
plt.yticks(np.arange(0, max_y ,10), fontsize=graph_data.fontsize) # step 10
plt.yscale('log',base=10) # or plt.yscale('linear')
ax.set_ylim(0.1, max_y)
## highlight selected y-label: https://stackoverflow.com/questions/73597796/make-one-y-axis-label-bold-in-matplotlib

# plotly
fig.update_layout(
      yaxis_range=[0,maxY],                   
      yaxis=dict(
          rangemode='tozero',  # Set the y-axis rangemode to 'tozero'
          dtick=maxY/10,
          gridcolor='rgb(196, 196, 196)', # grey
          gridwidth=1,
      ),
)

Bolden Contour Lines

1. Entire Figure

# mpl ?

# plotly: Add a rectangle shape to cover the entire subplot area
fig.add_shape(type="rect",
            xref="paper", yref="paper",
            x0=0, y0=0,
            x1=1, y1=1,
            line=dict(color="black", width=0.7))

2. and Each Bar Within

# mpl: Create a bar chart with bold outlines
plt.bar(categories, values, edgecolor='black', linewidth=2)

# plotly: ?

legend box

# mpl: 
ax.legend(loc='upper left', ncols=3, fontsize=graph_data.fontsize)
# legend out-of-figure, (1.02, 1) means anchor is upper right corner
plt.legend(loc='upper left', bbox_to_anchor=(1.02, 1), borderaxespad=0)

# Calculate the legend width based on the figure width
fig_size = plt.gcf().get_size_inches()
fig_width = fig_size[0]
# Move the legend to the center above the ceiling line
plt.legend(loc='upper center', bbox_to_anchor=(0.5, 1.1), 
  ncol=2,  # ncol=2 to have labels in one line
  frameon=False,  # frameon=False removes the legend box outline
  columnspacing=fig_width, # distance between each label
  handlelength=1.0, # label-box width (unit is text fontsize)
  handleheight=1.0, # label-box heigh (unit is text fontsize)
  prop={'size': 20, 'weight': 'bold'} # text fontsize          
) 

bar

# mpl: white hugo hatch with black bar edge.
# Problem: because the bug of mpl. hatch color follow the edge color
# Solved: draw bar twice, first the white hugo hatch with white bar edge. Second empty figure with black bar edge.
# white doubel ref: https://stackoverflow.com/questions/38168948/how-to-decouple-hatch-and-edge-color-in-matplotlib
# ref2: https://stackoverflow.com/questions/71424924/how-to-change-the-edge-color-of-markers-patches-in-matplotlib-legend
color_palette = ['black', (193/255, 1/255, 1/255), 'w', (127/255, 126/255, 127/255), 'blue']
pattern_list = ["", "/", "+", "\\", "x"]
edgecolor_list = ['w', 'w', (0/255, 176/255, 80/255), 'w', 'w']
    
ax.bar(x + offset, measurement, width, label=species_name,
                       color=color_palette[idx],
                        hatch = pattern_list[idx],
                        edgecolor=edgecolor_list[idx],
                        linewidth=1,
                     )
ax.bar(x + offset, measurement, width, label=species_name,
            color = "none",
            edgecolor='black', linewidth=1,
              )
# related legend: https://stackoverflow.com/questions/71424924/how-to-change-the-edge-color-of-markers-patches-in-matplotlib-legend
handles1, labels1 = ax.get_legend_handles_labels()
plt.legend([handles1[2*idx]+handles1[2*idx+1] for idx in range(group_count)], 
            [labels1[2*idx] for idx in range(group_count)],  
            loc='upper center', bbox_to_anchor=(0.5, 1.12), 
            ncol=group_count,  # have labels in one line
            frameon=False,
            # bbox_transform=plt.gcf().transFigure,
            columnspacing=legend_width, 
            # handlelength=1.0, 
            handleheight=1.2, 
            prop={'size': graph_data.fontsize,
                'weight': 'heavy'}
) 


# plotly: find the overflow
overflow_pattern = ["/" if y > maxY  else "" for y in entry[1]]
fig.add_bar(x=x,y=yList, 
    name=barName, 
    marker=dict(
        color=color_list[i],
        pattern_shape = overflow_pattern,
        line=dict(color='black', width=2)
        ),
    textfont=dict(size=graph_data.fontsize),
)
# legend 
legend_num = len( barDict.items())
fig.update_layout(barmode="relative", 

        # legend_title="Legend Title",
        legend = dict(
          entrywidthmode='fraction', # https://plotly.com/python/legend/
          entrywidth= 0.2,
          x=0.5 - 0.5 * legend_num * 0.2,        # Set x to 0.5 for the center
          y=1.2,       # Set y to a value greater than 1 to move it above the plot
          orientation="h",  # Display legend items in a single line
        ),
)

Out-box text

To draw symmetry chart, we need to special highlight the overflow bar number.

If the ancher point locate in the plot box, it’s easy to show text above the ceil line using textposition="bottom" like option. In the opposite scenario, plotly and mathplotlib all will hide the out-box text.

# plotly
fig.add_annotation(
            x=[x[0][i],x[1][i]],  # 注释的 x 坐标为 "bc"
            y=min(maxY,entry),  # 注释的 y 坐标为该列的最大值
            text=f"{entry:.2f}",  # 注释的文本内容
            # valign = "bottom", # text position in text box(default invisible)
            yanchor = "bottom", # text box position relative to anchor
            showarrow=False,  # 显示箭头
            # bgcolor="rgba(255, 255, 255, 0.8)",  # 注释框背景颜色
            font=dict(size=graph_data.fontsize+2)  # 注释文本字体大小
        )

# mathplotlib
# Create labels for overflowed values
for i, value in enumerate(values):
    if value > maxY:
        ax.annotate(f'Overflow: {value:.2f}', (i, maxY), ha='center', va='bottom', fontsize=12)

But mlb can write text out box.

ax.text(1, -1.6, 'Increasing', ha="center")

# first parameter is text, xy is the anchor point, xytext is the text, 
# xytext 2 xy is a relative distance
ax.annotate('yahaha', xy=(0, -0.1), xycoords='axes fraction', xytext=(1, -0.1))

out-box line

mathplotlib(mpl) can achieve this using ref, but there are few blogs about plotly.

# mpl: from 1*1 size full-graph (0.5,0.2) to point (1,0.8)
# transform=gcf().transFigure : gcf() stands for "get current figure," and .transFigure indicates that the coordinates provided in [0.5, 0.5], [0, 1] are in figure-relative coordinates. This means that the line's position is defined relative to the entire figure, not just the axes
# clip_on=False : This setting means that the line is not clipped at the edges of the axes. It allows the line to extend beyond the axes' boundaries.
from pylab import *  
plot([0.5, 1], [0.2, 0.8], color='lightgreen', linestyle='--', lw=1 ,transform=gcf().transFigure, clip_on=False)

# mpl: arrow from xy 2 xytext
# xycoords='figure fraction' to Add annotation to the figure (full graph)
ax.annotate('', xy=(0, -0.1), xycoords='axes fraction', xytext=(1, -0.1),\
arrowprops=dict(arrowstyle="->", color='violet'))

in-box line

# mpl:
ax.axhline(y=12, color='red', linestyle='--', label='Horizontal Line at y=12')
ax.axvline(x=3, color='green', linestyle='-.', label='Vertical Line at x=3')

# plotly ref: https://plotly.com/python/horizontal-vertical-shapes/
fig.add_vline(x=2.5, line_width=3, line_dash="dash", line_color="green") # dot
fig.add_hline(y=0.9)

实践

气泡图

二维无向图

教程

3D图

dash + plotly

如果防火墙是关闭的，你可以直接部署在external address上。使用docker也是可行的办法

app.run_server(debug=True, host='202.38.72.23')

3D 散点图，拟合曲面与网格图

实际案例

折线图

import matplotlib.pyplot as plt

X = [str(i) for i in metricValue]
Y = accuracyResult

# 设置图片大小
fig, ax = plt.subplots(figsize=(10, 6))  # 指定宽度为10英寸，高度为6英寸

plt.plot(X, Y, marker='o')
plt.xlabel('Threshold Percentage(%)', fontsize=12)  # 设置x轴标签字体大小为12
plt.ylabel('Average Execution Time of Static Method', fontsize=12)  # 设置y轴标签字体大小为12
plt.title('Tuning load store pressure', fontsize=14)  # 设置标题字体大小为14

for i in range(len(X)):
    plt.text(X[i], Y[i], Y[i], 
        fontsize=10, # 设置文本字体大小为10
        ha='center', # 设置水平对齐方式
        va='bottom')  # 设置垂直对齐方式

# 保存图片
plt.savefig(glv._get("resultPath") + f"tuning/{tuningLabel}/loadStorePressure.png", dpi=300)  # 设置dpi为300，可调整保存图片的分辨率

plt.show()  # 显示图片
plt.close()

Heatmap

实例

Stacked Bar & grouped compare bar

Example

Error Bars

在柱状图中，用于表示上下浮动的元素通常被称为“误差条”（Error Bars）。误差条是用于显示数据点或柱状图中的不确定性或误差范围的线条或线段。它们在柱状图中以垂直方向延伸，可以显示上下浮动的范围，提供了一种可视化的方式来表示数据的变化或不确定性。误差条通常通过标准差、标准误差、置信区间或其他统计指标来计算和表示数据的浮动范围。

Errorbars + StackedBars stacked 的过程中由于向上的error线的会被后面的Bar遮盖，然后下面的error线由于arrayminus=[i-j for i,j in zip(sumList,down_error)]导致大部分时间说负值，也不会显示。

fig = go.Figure()

 # color from https://stackoverflow.com/questions/68596628/change-colors-in-100-stacked-barchart-plotly-python
 color_list = ['rgb(29, 105, 150)', \
             'rgb(56, 166, 165)', \
    'rgb(15, 133, 84)',\
          'rgb(95, 70, 144)']
 sumList = [0 for i in range(len(x[0]))]
 for i, entry in enumerate( barDict.items()):
  barName=entry[0]
  yList = entry[1]
  ic(sumList,yList)
  sumList = [x + y for x, y in zip(yList, sumList)]
  fig.add_bar(x=x,y=yList, 
              name=barName, 
              text =[f'{val:.2f}' for val in yList], 
              textposition='inside',
              marker=dict(color=color_list[i]),
              error_y=dict(
    type='data',
    symmetric=False,
    color='purple',
    array=[i-j for i,j in zip(up_error,sumList)],
    arrayminus=[i-j for i,j in zip(sumList,down_error)],
       thickness=2, width=10),
              textfont=dict(size=8)
  )

Candlestick

类似股票上下跳动的浮标被称为”Candlestick”（蜡烛图）或”OHLC”（开盘-最高-最低-收盘）图表。

需要进一步的研究学习

暂无

遇到的问题

暂无

开题缘由、总结、反思、吐槽~~

参考文献

上面回答部分来自ChatGPT-3.5，暂时没有校验其可靠性(看上去貌似说得通)。

[1] Saket, B., Endert, A. and Demiralp, Ç., 2018. Task-based effectiveness of basic visualizations.IEEE transactions on visualization and computer graphics,25(7), pp.2505-2512.