SHAOJIE'S BOOK
HomeArchivesCategoriesTagsAbout
  • Tags
  • SP
Posted 2024-01-03Updated 2025-01-30Artificial Intelligence21 minutes read (About 3138 words)

AI Traning Parallism

导言

  • AI 训练时,有些分布式训练的常见并行概念需要了解。
    • 例如,TP, VP, SP, VPP
Read more
Shaojie Tan

Shaojie Tan

𝘊𝘰𝘮𝘱𝘶𝘵𝘦𝘳 𝘈𝘳𝘤𝘩𝘪𝘵𝘦𝘤𝘵𝘶𝘳𝘦 & 𝘏𝘗𝘊

Anhui, Hefei, China

Posts

440

Categories

35

Tags

527

Follow

Categories

  • Algorithms12
  • Architecture41
  • Artificial Intelligence19
  • Camp1
  • Databases2
  • HPC1
  • Math2
  • Network8
  • OOW31
  • Operating system8
  • Overview10
  • Programming31
  • ProjectRecord3
  • Software1
  • Thinking5
  • Tips9
  • Treasure1
  • Tutorials117
  • Values2
  • architecture2
  • diary4
  • english4
  • hardware2
  • love1
  • math3
  • network19
  • operating system5
  • programming1
  • security3
  • software23
  • thinking9
    • OOW1
  • tips5
  • toLearn52
  • values2

Subscribe for updates

follow.it

Links

  • ibugsibug.io
  • jiegecjia.je
  • leimaoleimao.github.io

Recents

2025-01-29

Future Plans: House & Car

OOW

2025-01-13

宛如泥潭的大型项目开发困境

Programming

2025-01-07

Continuous Integration, CI

software

2025-01-02

AI Model Visualization

Artificial Intelligence

2024-12-31

Naming

Programming

Archives

  • January 20254
  • December 20245
  • November 20241
  • October 20245
  • September 202411
  • August 20243
  • July 20245
  • June 20241
  • May 20242
  • April 20248
  • March 20243
  • February 20241
  • January 202414
  • December 202317
  • November 202333
  • October 202358
  • September 202336
  • August 202323
  • July 202324
  • June 20238
  • May 202319
  • April 20239
  • March 20235
  • February 20232
  • January 20236
  • December 20222
  • November 20224
  • October 20225
  • September 20224
  • August 20228
  • July 20227
  • June 20227
  • May 202210
  • April 20225
  • March 20227
  • February 20222
  • January 20225
  • December 20214
  • November 20212
  • October 202113
  • September 202111
  • August 202113
  • July 202128

Tags

5G1
64bits vs 32bits1
AI3
AMAT1
AMD3
ASPLOS1
ATI1
AVX2
Algorithm1
Alpha1
Analysis1
Apt1
Assembly4
BFS1
BHive1
BT3
BTL1
Baka Mitai1
Bash1
Big-Endian1
SHAOJIE'S BOOK

© 2025 Shaojie Tan  Powered by Hexo & Icarus

© 2019

×