WEBVTT

1
00:00:00.147 --> 00:00:05.200
<v Cassidy>A <00:00:00.213>multi-university <00:00:01.107>paper <00:00:01.520>says <00:00:01.860>the <00:00:01.971>benchmarks <00:00:02.520>can <00:00:02.667>be <00:00:02.789>cooked — <00:00:03.780>and <00:00:03.860>the <00:00:03.968>very <00:00:04.224>same <00:00:04.544>week

2
00:00:05.264 --> 00:00:10.120
<v Cassidy>Meta <00:00:05.570>decides <00:00:05.940>the <00:00:06.080>cure <00:00:06.453>is <00:00:06.613>to <00:00:06.768>hand <00:00:06.980>you <00:00:07.060>the <00:00:07.180>weights <00:00:07.640>and <00:00:07.800>let <00:00:07.940>you <00:00:08.040>check <00:00:08.260>the <00:00:08.384>math <00:00:08.702>yourself.

3
00:00:11.168 --> 00:00:13.920
<v Cassidy>This <00:00:11.253>is <00:00:11.380>the <00:00:11.573>AI <00:00:11.893>Daily <00:00:12.200>Briefing <00:00:12.600>for <00:00:12.792>Wednesday.

4
00:00:14.000 --> 00:00:19.360
<v Cassidy>Today: <00:00:14.650>the <00:00:14.856>Princeton-MIT <00:00:16.000>paper <00:00:16.440>putting <00:00:16.800>every <00:00:17.280>“state <00:00:17.547>of <00:00:17.640>the <00:00:17.800>art” <00:00:18.053>claim <00:00:18.373>under <00:00:18.680>a <00:00:18.800>shadow

5
00:00:19.504 --> 00:00:23.200
<v Cassidy>Meta’s <00:00:20.224>open-source <00:00:20.864>move <00:00:21.140>through <00:00:21.300>the <00:00:21.416>inference-stack <00:00:22.336>lens

6
00:00:23.220 --> 00:00:27.040
<v Cassidy>and <00:00:23.360>HubSpot’s <00:00:24.080>200 <00:00:24.540>updates <00:00:24.992>with <00:00:25.227>no <00:00:25.488>technical <00:00:25.977>report <00:00:26.373>in <00:00:26.547>sight.

7
00:00:27.679 --> 00:00:36.879
<v Bill>When <00:00:27.812>an <00:00:28.026>AI <00:00:28.299>company <00:00:28.799>drops <00:00:29.239>a <00:00:29.375>blog <00:00:29.871>post <00:00:30.422>saying <00:00:31.079>their <00:00:31.339>new <00:00:31.599>model <00:00:32.106>is <00:00:32.559>“state <00:00:33.146>of <00:00:33.239>the <00:00:33.439>art” <00:00:33.979>because <00:00:34.426>it <00:00:34.570>topped <00:00:35.199>some <00:00:35.591>benchmark

8
00:00:36.919 --> 00:00:44.223
<v Bill>how <00:00:37.092>do <00:00:37.226>we <00:00:37.323>actually <00:00:37.759>know <00:00:38.132>if <00:00:38.287>that <00:00:38.525>number <00:00:38.852>means <00:00:39.181>anything — <00:00:40.266>or <00:00:40.559>if <00:00:40.767>they <00:00:41.039>just <00:00:41.409>figured <00:00:41.819>out <00:00:42.059>how <00:00:42.292>to <00:00:42.539>ace <00:00:43.019>the <00:00:43.327>test?

9
00:00:44.223 --> 00:00:51.599
<v Cassidy>Yeah, <00:00:45.172>it’s <00:00:45.339>a <00:00:45.423>genuinely <00:00:46.039>important <00:00:46.470>question, <00:00:47.539>and <00:00:47.619>the <00:00:47.732>short <00:00:48.045>answer <00:00:48.479>is: <00:00:49.159>a <00:00:49.479>lot <00:00:49.626>of <00:00:49.719>those <00:00:49.969>numbers <00:00:50.359>don’t <00:00:50.591>mean <00:00:50.767>much <00:00:51.029>anymore.

10
00:00:51.959 --> 00:00:55.359
<v Cassidy>A <00:00:52.372>paper <00:00:52.687>from <00:00:52.919>a <00:00:53.012>multi-university <00:00:54.047>team — <00:00:54.847>Princeton

11
00:00:55.519 --> 00:01:02.799
<v Cassidy>MIT <00:00:56.219>Cornell <00:00:56.872>Brown <00:00:57.139>and <00:00:57.302>others — <00:00:58.199>published <00:00:58.612>on <00:00:58.826>arXiv <00:00:59.567>puts <00:00:59.786>it <00:00:59.906>bluntly: <00:01:00.655>benchmarking <00:01:01.652>is <00:01:01.839>broken.

12
00:01:02.919 --> 00:01:04.879
<v Cassidy>The <00:01:03.329>problem <00:01:03.759>has <00:01:04.039>two <00:01:04.266>parts.

13
00:01:05.026 --> 00:01:10.159
<v Cassidy>First, <00:01:06.457>saturation. <00:01:07.655>According <00:01:08.186>to <00:01:08.310>analysis <00:01:08.911>from <00:01:09.119>Benchr

14
00:01:10.207 --> 00:01:14.319
<v Cassidy>most <00:01:10.426>of <00:01:10.519>the <00:01:10.690>benchmarks <00:01:11.239>you <00:01:11.419>see <00:01:11.666>cited <00:01:12.063>most <00:01:12.399>often — <00:01:13.479>MMLU

15
00:01:14.426 --> 00:01:23.439
<v Cassidy>HumanEval <00:01:15.719>GSM8K — <00:01:17.471>have <00:01:17.656>scores <00:01:18.063>clustered <00:01:18.746>so <00:01:19.039>high <00:01:19.391>that <00:01:19.539>the <00:01:19.679>top <00:01:19.983>five <00:01:20.292>frontier <00:01:20.776>models <00:01:21.139>are <00:01:21.255>separated <00:01:21.812>by <00:01:21.967>less <00:01:22.191>than <00:01:22.372>three <00:01:22.730>points.

16
00:01:23.759 --> 00:01:31.679
<v Cassidy>A <00:01:24.115>tournament <00:01:24.519>where <00:01:24.772>everyone <00:01:25.256>scores <00:01:25.626>in <00:01:25.699>the <00:01:25.799>upper <00:01:26.059>nineties <00:01:26.739>has, <00:01:27.146>in <00:01:27.226>their <00:01:27.439>words, <00:01:28.119>“stopped <00:01:28.709>sorting.” <00:01:29.530>Second, <00:01:30.845>gaming.

17
00:01:31.879 --> 00:01:34.479
<v Cassidy>Per <00:01:32.511>that <00:01:32.703>same <00:01:33.019>Benchr <00:01:33.448>analysis

18
00:01:34.559 --> 00:01:43.439
<v Cassidy>HumanEval <00:01:35.419>and <00:01:35.719>MMLU <00:01:36.519>are <00:01:36.688>specifically <00:01:37.349>flagged <00:01:37.866>as <00:01:37.977>benchmarks <00:01:38.586>to <00:01:38.742>retire <00:01:39.439>because <00:01:40.015>they’ve <00:01:40.191>been <00:01:40.452>so <00:01:40.772>extensively <00:01:41.409>trained <00:01:41.829>against <00:01:42.415>that <00:01:42.859>a <00:01:42.943>high <00:01:43.106>score

19
00:01:43.499 --> 00:01:49.279
<v Cassidy>may <00:01:43.711>tell <00:01:43.899>you <00:01:44.127>more <00:01:44.359>about <00:01:44.589>whether <00:01:44.839>a <00:01:44.929>company <00:01:45.343>optimized <00:01:45.959>for <00:01:46.099>the <00:01:46.271>test <00:01:46.847>than <00:01:47.379>whether <00:01:47.559>the <00:01:47.732>model <00:01:48.052>is <00:01:48.203>actually <00:01:48.629>capable.

20
00:01:49.599 --> 00:01:54.079
<v Cassidy>So <00:01:50.255>when <00:01:50.359>you <00:01:50.519>see <00:01:50.679>a <00:01:50.763>headline <00:01:51.186>score, <00:01:51.699>ask: <00:01:52.399>is <00:01:52.751>this <00:01:52.927>benchmark <00:01:53.423>saturated?

21
00:01:54.279 --> 00:01:58.079
<v Cassidy>Was <00:01:54.899>the <00:01:55.012>methodology <00:01:55.655>published, <00:01:56.259>and <00:01:56.346>could <00:01:56.559>a <00:01:56.666>third <00:01:56.852>party <00:01:57.175>reproduce <00:01:57.759>it?

22
00:01:58.279 --> 00:02:00.922
<v Cassidy>And <00:01:58.932>is <00:01:59.059>the <00:01:59.169>company <00:01:59.559>grading <00:01:59.899>its <00:02:00.119>own <00:02:00.301>homework?

23
00:02:00.922 --> 00:02:09.439
<v Bill>That <00:02:01.279>last <00:02:01.659>one — <00:02:02.215>companies <00:02:02.779>grading <00:02:03.226>their <00:02:03.419>own <00:02:03.688>homework — <00:02:04.815>what <00:02:04.911>does <00:02:05.071>that <00:02:05.261>actually <00:02:05.823>look <00:02:06.127>like <00:02:06.372>in <00:02:06.559>practice, <00:02:07.749>and <00:02:08.159>why <00:02:08.479>is <00:02:08.666>it <00:02:08.759>a <00:02:08.869>problem?

24
00:02:10.514 --> 00:02:19.394
<v Cassidy>The <00:02:10.674>arXiv <00:02:11.207>paper <00:02:11.554>flags <00:02:11.986>this <00:02:12.176>directly <00:02:12.727>in <00:02:12.874>its <00:02:13.061>title: <00:02:13.914>“Don’t <00:02:14.414>Let <00:02:14.727>AI <00:02:15.047>Be <00:02:15.194>Its <00:02:15.354>Own <00:02:15.567>Judge” — <00:02:16.794>meaning <00:02:17.106>when <00:02:17.274>a <00:02:17.367>model <00:02:17.661>is <00:02:17.786>evaluated <00:02:18.487>using <00:02:18.794>another <00:02:19.181>AI

25
00:02:19.451 --> 00:02:20.754
<v Cassidy>system <00:02:19.847>as <00:02:19.974>the <00:02:20.114>scorer

26
00:02:20.874 --> 00:02:25.394
<v Cassidy>the <00:02:21.285>incentives <00:02:21.834>get <00:02:22.043>circular <00:02:22.854>and <00:02:23.174>the <00:02:23.294>results <00:02:23.754>get <00:02:23.939>unreliable <00:02:24.722>fast.

27
00:02:25.687 --> 00:02:33.074
<v Cassidy>So <00:02:26.327>watch <00:02:26.624>whether <00:02:26.874>a <00:02:26.970>benchmark <00:02:27.527>score <00:02:27.847>comes <00:02:28.146>with <00:02:28.327>an <00:02:28.487>independent <00:02:29.147>methodology — <00:02:30.386>something <00:02:30.754>a <00:02:30.874>third <00:02:31.114>party <00:02:31.354>could <00:02:31.610>reproduce <00:02:32.154>without <00:02:32.503>access <00:02:32.941>to <00:02:33.014>the

28
00:02:33.124 --> 00:02:34.514
<v Cassidy>company’s <00:02:33.598>internal <00:02:34.050>data.

29
00:02:34.754 --> 00:02:37.914
<v Cassidy>If <00:02:35.274>the <00:02:35.442>only <00:02:35.691>source <00:02:36.087>is <00:02:36.214>the <00:02:36.314>company’s <00:02:36.714>own <00:02:36.914>blog <00:02:37.314>post

30
00:02:37.970 --> 00:02:39.874
<v Cassidy>with <00:02:38.274>no <00:02:38.480>linked <00:02:38.802>technical <00:02:39.303>report

31
00:02:39.965 --> 00:02:42.674
<v Cassidy>system <00:02:40.594>card <00:02:41.074>or <00:02:41.216>external <00:02:41.781>replication

32
00:02:42.781 --> 00:02:46.434
<v Cassidy>treat <00:02:43.334>the <00:02:43.461>score <00:02:43.874>as <00:02:44.074>marketing <00:02:44.530>copy <00:02:44.954>until <00:02:45.194>someone <00:02:45.522>else <00:02:45.734>can <00:02:45.847>check <00:02:46.194>it.

33
00:02:47.110 --> 00:02:48.430
<v Cassidy>This <00:02:47.310>one's <00:02:47.542>from <00:02:47.730>Gizmodo:

34
00:02:48.785 --> 00:02:54.705
<v Quote>While <00:02:48.905>the <00:02:49.065>new <00:02:49.247>releases <00:02:49.793>will <00:02:50.043>reportedly <00:02:50.696>maintain <00:02:51.313>some <00:02:51.565>proprietary <00:02:52.292>parts <00:02:52.645>for <00:02:52.775>alleged <00:02:53.322>safety <00:02:53.736>purposes

35
00:02:54.725 --> 00:02:57.745
<v Quote>the <00:02:54.825>company <00:02:55.156>apparently <00:02:55.718>plans <00:02:56.038>to <00:02:56.209>open-source <00:02:56.805>the <00:02:56.945>models

36
00:02:57.802 --> 00:03:03.425
<v Quote>likely <00:02:58.189>offering <00:02:58.609>licensing <00:02:59.221>agreements <00:02:59.665>to <00:02:59.878>firms <00:03:00.177>that <00:03:00.337>want <00:03:00.492>to <00:03:00.605>use <00:03:00.825>the <00:03:00.998>model <00:03:01.315>instead <00:03:01.692>of <00:03:01.785>going <00:03:02.049>full <00:03:02.358>black <00:03:02.765>box

37
00:03:03.457 --> 00:03:04.865
<v Quote>like <00:03:03.633>many <00:03:03.852>of <00:03:03.945>its <00:03:04.125>competitors.

38
00:03:05.370 --> 00:03:12.030
<v Bill>Six <00:03:05.590>hundred <00:03:05.940>billion <00:03:06.510>earmarked <00:03:07.130>for <00:03:07.550>AI, <00:03:08.610>and <00:03:08.710>the <00:03:08.874>headline <00:03:09.310>move <00:03:09.817>is <00:03:10.302>open-sourcing <00:03:11.330>the <00:03:11.460>weights.

39
00:03:12.206 --> 00:03:14.270
<v Bill>Hard <00:03:12.937>to <00:03:13.054>call <00:03:13.358>that <00:03:13.603>floundering.

40
00:03:14.457 --> 00:03:17.950
<v Bill>Feels <00:03:15.038>like <00:03:15.294>Meta <00:03:15.630>just <00:03:16.007>showed <00:03:16.403>us <00:03:16.550>the <00:03:16.750>strategy.

41
00:03:17.970 --> 00:03:20.510
<v Cassidy>And <00:03:18.090>Gizmodo <00:03:18.550>wants <00:03:18.770>you <00:03:18.870>reading <00:03:19.170>the <00:03:19.310>“floundering” <00:03:20.110>word.

42
00:03:20.830 --> 00:03:29.030
<v Cassidy>I’d <00:03:21.262>read <00:03:21.417>it <00:03:21.490>through <00:03:21.650>the <00:03:21.766>inference <00:03:22.323>stack <00:03:22.660>instead: <00:03:23.998>open <00:03:24.240>weights <00:03:24.622>mean <00:03:24.810>nothing <00:03:25.283>if <00:03:25.470>Meta <00:03:25.817>still <00:03:26.021>controls <00:03:26.610>who <00:03:26.750>fine-tunes <00:03:27.570>and <00:03:27.670>who <00:03:27.859>serves <00:03:28.297>at <00:03:28.430>scale.

43
00:03:29.030 --> 00:03:35.630
<v Bill>Right <00:03:30.070>and <00:03:30.250>tie <00:03:30.483>it <00:03:30.638>back <00:03:30.883>to <00:03:31.022>what <00:03:31.230>we <00:03:31.454>just <00:03:31.750>hit <00:03:31.950>on <00:03:32.212>benchmarks — <00:03:33.710>if <00:03:33.910>the <00:03:34.094>yardsticks <00:03:34.690>are <00:03:34.830>bent

44
00:03:35.674 --> 00:03:39.390
<v Bill>shipping <00:03:36.110>open <00:03:36.480>weights <00:03:36.883>is <00:03:37.030>one <00:03:37.177>of <00:03:37.270>the <00:03:37.490>few <00:03:37.870>honest <00:03:38.417>moves <00:03:38.846>left.

45
00:03:39.570 --> 00:03:43.070
<v Bill>The <00:03:40.160>outside <00:03:40.563>world <00:03:40.894>gets <00:03:41.177>to <00:03:41.290>run <00:03:41.530>its <00:03:41.750>own <00:03:41.998>evals.

46
00:03:43.097 --> 00:03:50.670
<v Cassidy>If <00:03:43.182>they <00:03:43.358>ship <00:03:43.643>clean. <00:03:44.560>The <00:03:44.887>report <00:03:45.260>already <00:03:45.534>says <00:03:45.838>some <00:03:46.083>parts <00:03:46.382>stay <00:03:46.577>proprietary <00:03:47.370>for <00:03:47.670>“safety.” <00:03:48.441>Always <00:03:49.490>the <00:03:49.630>asterisk.

47
00:03:50.723 --> 00:03:56.590
<v Bill>First <00:03:51.040>release <00:03:51.443>under <00:03:51.800>Alexandr <00:03:52.638>Wang, <00:03:53.763>fresh <00:03:54.110>off <00:03:54.390>the <00:03:54.590>Scale <00:03:55.203>AI <00:03:55.703>acquisition.

48
00:03:56.883 --> 00:04:05.023
<v Bill>So <00:03:57.530>the <00:03:57.790>data-labeling <00:03:58.590>guy <00:03:58.883>is <00:03:59.050>now <00:03:59.280>running <00:03:59.650>the <00:03:59.810>lab <00:04:00.142>that <00:04:00.316>bought <00:04:00.630>his <00:04:00.810>company <00:04:01.310>to <00:04:01.570>fix <00:04:01.930>its <00:04:02.190>models. <00:04:03.790>No <00:04:04.163>pressure.

49
00:04:05.023 --> 00:04:10.110
<v Cassidy>And <00:04:05.350>the <00:04:05.516>timing <00:04:05.857>lands <00:04:06.230>the <00:04:06.398>same <00:04:06.622>week <00:04:06.790>the <00:04:06.956>Senate <00:04:07.264>killed <00:04:07.490>the <00:04:07.590>federal <00:04:07.930>moratorium <00:04:08.647>ninety-nine <00:04:09.417>to <00:04:09.710>one.

50
00:04:10.330 --> 00:04:13.390
<v Cassidy>You <00:04:11.006>want <00:04:11.097>to <00:04:11.200>compete <00:04:11.590>under <00:04:11.843>fifty <00:04:12.134>different <00:04:12.403>state <00:04:12.740>regimes?

51
00:04:13.518 --> 00:04:19.070
<v Cassidy>Open-source <00:04:14.450>the <00:04:14.577>model <00:04:14.950>and <00:04:15.110>let <00:04:15.300>licensees <00:04:15.990>carry <00:04:16.210>the <00:04:16.350>fine-tuning <00:04:17.150>risk. <00:04:17.710>Very <00:04:18.408>convenient.

52
00:04:19.671 --> 00:04:20.791
<v Cassidy>This <00:04:19.791>one's <00:04:20.023>from <00:04:20.211>HubSpot:

53
00:04:21.648 --> 00:04:29.008
<v Quote>“SMBs <00:04:22.328>don't <00:04:22.656>need <00:04:22.912>more <00:04:23.301>AI <00:04:23.648>hype—they <00:04:24.480>need <00:04:24.666>technology <00:04:25.264>that <00:04:25.395>helps,” <00:04:25.872>said <00:04:26.192>Andy <00:04:26.528>Pitre, <00:04:27.368>EVP <00:04:27.781>of <00:04:27.938>Product <00:04:28.341>at <00:04:28.488>HubSpot.

54
00:04:29.448 --> 00:04:37.568
<v Quote>“The <00:04:30.004>products <00:04:30.341>we’re <00:04:30.568>launching <00:04:30.955>in <00:04:31.028>our <00:04:31.145>Spring <00:04:31.648>2025 <00:04:32.512>Spotlight <00:04:33.128>are <00:04:33.288>helping <00:04:33.621>teams <00:04:33.952>move <00:04:34.288>fast <00:04:34.661>on <00:04:34.981>AI <00:04:35.468>and <00:04:35.715>solve <00:04:36.061>their <00:04:36.208>go-to-market <00:04:36.826>challenges.

55
00:04:37.781 --> 00:04:44.608
<v Quote>We've <00:04:38.341>embedded <00:04:38.875>AI <00:04:39.132>throughout <00:04:39.608>our <00:04:39.797>entire <00:04:40.288>platform <00:04:41.061>so <00:04:41.372>businesses <00:04:41.861>of <00:04:42.008>any <00:04:42.224>size <00:04:42.648>can <00:04:42.808>start <00:04:43.054>seeing <00:04:43.408>value <00:04:43.868>immediately

56
00:04:44.688 --> 00:04:46.928
<v Quote>without <00:04:45.288>massive <00:04:45.648>teams <00:04:46.128>or <00:04:46.358>budgets.”

57
00:04:47.165 --> 00:04:57.245
<v Bill>Two <00:04:47.375>hundred-plus <00:04:48.267>features, <00:04:49.156>embedded <00:04:49.938>AI <00:04:50.434>across <00:04:50.885>the <00:04:51.058>whole <00:04:51.414>platform, <00:04:52.645>and <00:04:52.785>the <00:04:52.912>pitch <00:04:53.378>is <00:04:53.925>SMBs <00:04:55.037>feel <00:04:55.549>like <00:04:55.837>they’re <00:04:56.105>falling <00:04:56.616>behind.

58
00:04:58.072 --> 00:05:01.165
<v Bill>So <00:04:58.165>the <00:04:58.365>cure <00:04:58.725>for <00:04:59.005>AI <00:04:59.397>overwhelm <00:05:00.205>is...

59
00:05:01.205 --> 00:05:03.258
<v Bill>two <00:05:01.365>hundred <00:05:01.725>more <00:05:02.091>things <00:05:02.418>to <00:05:02.592>learn?

60
00:05:03.258 --> 00:05:05.885
<v Cassidy>And <00:05:03.885>notice <00:05:04.237>what <00:05:04.418>comes <00:05:04.701>with <00:05:04.797>this <00:05:04.978>press <00:05:05.305>release.

61
00:05:06.837 --> 00:05:14.966
<v Cassidy>Nothing. <00:05:07.458>No <00:05:07.773>technical <00:05:08.296>report, <00:05:09.058>no <00:05:09.421>eval, <00:05:09.938>no <00:05:10.095>failure <00:05:10.525>rate — <00:05:11.357>just <00:05:11.525>a <00:05:11.618>count <00:05:11.978>of <00:05:12.196>features <00:05:12.465>and <00:05:12.565>a <00:05:12.658>quote <00:05:12.957>from <00:05:13.105>the <00:05:13.305>EVP <00:05:13.778>of <00:05:13.955>Product.

62
00:05:14.966 --> 00:05:21.965
<v Bill>Right, <00:05:15.765>and <00:05:15.905>the <00:05:16.034>honest <00:05:16.495>version <00:05:16.872>of <00:05:16.973>this <00:05:17.218>is <00:05:17.469>fine — <00:05:18.381>they’re <00:05:18.578>automating <00:05:19.105>the <00:05:19.232>stuff <00:05:19.605>a <00:05:19.725>sales <00:05:20.185>rep <00:05:20.445>already <00:05:21.072>hated <00:05:21.485>doing.

63
00:05:22.085 --> 00:05:25.325
<v Bill>Logging <00:05:23.045>calls, <00:05:24.018>drafting <00:05:24.502>follow-ups.

64
00:05:26.275 --> 00:05:34.334
<v Bill>I <00:05:26.317>want <00:05:26.472>to <00:05:26.557>know <00:05:26.715>whether <00:05:27.005>any <00:05:27.298>of <00:05:27.445>those <00:05:27.705>two <00:05:27.925>hundred <00:05:28.275>updates <00:05:28.835>changes <00:05:29.357>what <00:05:29.525>the <00:05:29.705>rep <00:05:29.947>actually <00:05:30.525>has <00:05:30.818>to <00:05:31.021>know, <00:05:32.018>or <00:05:32.173>just <00:05:32.445>kills <00:05:32.865>the <00:05:33.005>busywork.

65
00:05:34.334 --> 00:05:42.605
<v Cassidy>Andy <00:05:34.985>Pitre <00:05:35.325>says <00:05:35.765>SMBs <00:05:36.305>don’t <00:05:36.477>need <00:05:36.637>more <00:05:36.845>hype, <00:05:37.661>they <00:05:37.757>need <00:05:37.949>tech <00:05:38.237>that <00:05:38.445>helps — <00:05:39.533>said <00:05:39.752>in <00:05:39.845>a <00:05:39.935>release <00:05:40.329>announcing <00:05:40.845>two <00:05:41.125>hundred <00:05:41.458>features <00:05:41.912>at <00:05:42.077>once.

66
00:05:43.453 --> 00:05:44.778
<v Cassidy>Pick <00:05:43.525>a <00:05:43.629>lane, <00:05:44.045>Andy.

67
00:05:44.778 --> 00:05:50.605
<v Bill>And <00:05:45.325>after <00:05:45.565>the <00:05:45.725>benchmark <00:05:46.425>segment <00:05:46.898>we <00:05:47.053>just <00:05:47.345>did, <00:05:48.298>there’s <00:05:48.578>no <00:05:48.742>number <00:05:49.149>here <00:05:49.458>to <00:05:49.629>even <00:05:49.978>check.

68
00:05:50.898 --> 00:05:55.085
<v Bill>At <00:05:51.525>least <00:05:51.765>a <00:05:51.858>frontier <00:05:52.365>lab <00:05:52.658>gives <00:05:52.945>you <00:05:53.085>a <00:05:53.211>cooked <00:05:53.549>benchmark <00:05:54.178>to <00:05:54.352>argue <00:05:54.738>about.

69
00:05:55.245 --> 00:05:57.085
<v Bill>This <00:05:55.912>is <00:05:56.013>just <00:05:56.245>a <00:05:56.335>feature <00:05:56.752>count.

70
00:05:57.653 --> 00:06:02.613
<v Cassidy>If <00:05:57.866>AI <00:05:58.186>Daily <00:05:58.493>Briefing <00:05:58.880>is <00:05:58.981>part <00:05:59.200>of <00:05:59.285>your <00:05:59.473>routine, <00:06:00.337>consider <00:06:00.753>subscribing <00:06:01.457>wherever <00:06:01.833>you’re <00:06:02.029>listening.

71
00:06:02.733 --> 00:06:07.813
<v Cassidy>And <00:06:03.120>if <00:06:03.193>you <00:06:03.285>have <00:06:03.453>a <00:06:03.550>moment, <00:06:04.240>leave <00:06:04.413>a <00:06:04.493>quick <00:06:04.773>review — <00:06:05.680>it <00:06:05.779>really <00:06:06.093>helps <00:06:06.333>other <00:06:06.579>people <00:06:06.901>find <00:06:07.113>the <00:06:07.253>show.

72
00:06:08.601 --> 00:06:11.721
<v Cassidy>We’ve <00:06:08.801>put <00:06:08.941>the <00:06:09.068>links <00:06:09.421>for <00:06:09.525>everything <00:06:09.988>we <00:06:10.081>covered <00:06:10.414>today <00:06:10.708>in <00:06:10.781>the <00:06:10.889>show <00:06:11.174>notes.

73
00:06:11.908 --> 00:06:15.721
<v Cassidy>If <00:06:12.313>something <00:06:12.635>caught <00:06:12.857>your <00:06:13.021>ear, <00:06:13.849>that’s <00:06:14.141>the <00:06:14.241>place <00:06:14.468>to <00:06:14.581>dig <00:06:14.788>in <00:06:14.881>a <00:06:14.955>little <00:06:15.218>deeper.

74
00:06:15.881 --> 00:06:18.641
<v Cassidy>That’s <00:06:16.708>AI <00:06:16.961>Daily <00:06:17.214>Briefing <00:06:17.601>for <00:06:17.801>today.

75
00:06:18.729 --> 00:06:20.601
<v Cassidy>This <00:06:19.108>is <00:06:19.201>a <00:06:19.301>Lantern <00:06:19.821>Podcast.