Frontier Silicon and Massive Models Reshape the AI Landscape

• M6's new CPU complex splits into 2 super cores, 4 performance cores, and 6 efficiency cores, delivering up to 1.2x faster multithreaded performance than the M5.
• The M5 Ultra utilizes a first-of-its-kind quad-die design that connects two dual-die M5 Max chips via UltraFusion, achieving an inter-die bandwidth exceeding 4.4TB/s.
• M5 Ultra supports up to 512GB of unified memory with 1.2TB/s of bandwidth, enabling the local execution of LLMs with hundreds of billions of parameters without cloud dependency.
• Both chips integrate Neural Accelerators directly into their GPU cores, boosting peak AI compute by nearly 30% on the M6 and up to 4.5x on the M5 Ultra compared to their predecessors.
• The M5 Ultra's media engine includes hardware-accelerated AV1 decode and four dedicated ProRes encode/decode engines, while Apple Intelligence features are slated for release with macOS 27 this fall.

I was blown away by M1 Pro, and used it for 4 years before it became a little sluggish, and replacing it with an Asus S16 running on Ryzen AI 9 HX 370 (an excellent laptop). One of the main reasons was missing my old Linux setup.

I've briefly tested M5 Pro in an Apple store and was surprised by how quick it felt and did anything. A tangible and significant difference, and I really feel it would be good getting it, or M6.

However - before, MacOS was a big factor driving me towards Apple - now, it would be hard for me to give up my Linux setup and all the stuff I love about it, even for such performance...Linux progressed really nicely, while MacOS deteriorated at the same rate, and it's changing the balance and the decision for me.

— alluro2 (thread)

Weird, I'm still using my M1 Pro and it feels just as fast as it always has. MacOS feels almost identical to me as it did 5 years ago, too, except the ugly icon change.

— boredtofears (thread)

Damn. I just bought a maxed out MacBook Pro M5 Max 128GB 8TB, still waiting for it to be delivered. I could get 256GB RAM M5 Ultra 1TB for roughly the same price, and it's double the memory bandwidth. Which one would you recommend? I do plan to run local LLMs.

— logotype (thread)

• The model has 125B total parameters with only 6B active per token, making it highly efficient for inference.
• It incorporates 51B of n-grams and a new attention mechanism called Qwen Sparse Attention.
• The release is explicitly described as a preview of the next-generation Qwen4 architecture.
• The model is aimed at Mac users, Strix Halo laptops, and DGX Spark systems for local deployment.

I was already rolling around the idea of a 128GB M5 Max MBP. Now this!

A 4-bit MLX quant with 128k window should fit perfectly, in the 50-70 tok/s range.

— pwython (thread)

I enjoy the Qwen models a lot, but building things on top of them with OpenRouter has been painful.

OpenRouter does a lot of great work and I really enjoy being able to use different models so easily. I like when a provider is phasing out an older model that still works for my needs and the price is much lower. It seems like such a good win-win.

However, the problem is that many Qwen models have almost no capacity or is so flaky you literally have to just litter your code with a blacklist/whitelist of providers. OpenRouter has some attempts to solve this, but they don't work. In fact, OpenRouter has a lot of really cool stuff that is documented, but if you read the code it's not yet implemented or isn't actually there yet, which is a shame.

— ddtaylor (thread)

Finally, a reason to own a 128GB Strix Halo or GB10 device. Or a reason to consider the new Mac Studio.

I have a Strix Halo and dual 32GB GPUs in my desktop, and the latter is pretty much always better for running local models because it's quite a bit faster due to higher memory bandwidth. There simply haven't been any models that are better than Qwen 27B or Gemma 31B, which run comfortably in 64GB with big context.

— SwellJoe (thread)

• Jalapeño's B0 stepping delivers 13.4 PFLOPs of MXFP4 compute on TSMC's N3P process with a 700W TDP, outperforming Rubin's density despite lower power limits.
• The chip rejects prefill-decode disaggregation, opting for a homogenous pool that prevents hardware stranding when input-to-output token ratios fluctuate.
• OpenAI's internal Gluon programming language and a scaled-up Codex AI autonomously generate and tune hand-written kernels, yielding over 2x throughput improvements in under two weeks without human engineer intervention.
• A single Jalapeño rack contains 128 ASICs across 16 trays and connects to a global scale-up domain of up to 2,048 XPUs using Tomahawk 6 switches and optical circuit switches.

I love how now you have to consider the possible s** posting motivation behind analysis of a trillion dollar industry being conducted at a world-class level by a bunch of ex Reddit and 4Chan adjacent mods -- it's one of the best stories in AI that SemiAnalysis is not cut from the same cloth as Gartner McKinsey et al

— jimmySixDOF (thread)

I hadn't seen the token/Joules comparison with human speech before. Humans are still 22x more efficient, which is not that far considering the rate of progress in this area.

— fraboniface (thread)

I don't agree. At the moment companies like NVIDIA take several times what it costs to make a chip. I think the fair split for the technology contribution is more like 50-50, maybe even 30-70 in favour of the manufacturer.

With competition we will actually have the fair split, whatever that is, and thus much lower prices.

At the moment, to have a big AI firm, or really AI firm at all, you need to be blessed by NVIDIA, in the form of receiving circular financing for your compute. They know that their prices aren't fair, or competitive.

Commoditization of inference is the end of that. The end of the mega-premium on inference hardware, and it's good not only for people who like running their LLMs, but it's the first step towards commoditization of training.

— impossiblefork (thread)

• AI occupied four out of the top five spots on February 4 and February 12, and arguably all five on February 5.
• In June 2026, the first half of the month saw roughly 60% of the daily lineup dedicated to AI topics or generation, before tapering to approximately 50% by month's end.
• Only three days in February featured no large language model news within the top five stories, with AI first appearing as low as the number seven or eight spot on other days.
• A flagged AI-written submission titled 'AI is not a coworker, it's an exoskeleton' garnered over 500 upvotes and 500 comments, illustrating the platform's receptiveness to AI-generated discourse.

Democratization of access has its victims.

I miss the old newsgroups. I wish we could land again in that weird, colorful oceans of geeky individuals with their quirks and interesting nerdy takes.

15 years ago, HN was about bootstrapping and early YC stars.

10 years ago, HN was in full VC-enterprise sauce.

5 years ago, it was crypto and NFTs, and how I switch jobs every quarter.

Now it is a blurry view into the future through the black mirror of AI.

How could we jump back into the cozy past again?

I do not have a clear answer, but what seems to work for you are small groups of friends and friends of friends hanging out on WhatsApp or Discord, discussing geeky news together in the context of their backgrounds. But these are very curated and very closed groups, still operating on the social media platforms of behemoths.

Would now be the right time to restart self-hosted message boards or Mattermost/Campfire instances?

— mgl (thread)

I joined HN 13 years ago (2 years after you), and back then I saw it as a bunch of libertarians and venture capitalists and some open source (and/or functional programming) enthusiast with a few interesting and nerdy projects. I was mostly here for the news about new and useful technology.

I think the visual weight of the first group has increased over time as their projects of making a bunch of money while offering nothing of value has been validated more and more by our capitalist economy.

— runarberg (thread)

I never cared much for newsgroups, they were mostly nice for the historical stuff. I do miss IRC though, as toxic as it could be. I could run it in a little terminal on any system without download electron apps and being spammed with ads (discord).

— datakan (thread)

• The training pipeline uses state-of-the-art mid-training and post-training techniques to specialize the base model rather than relying on brute-scale compute.
• Thomson demonstrates a measurable performance uplift in executing complex, multi-part professional instructions compared to its open-source baseline.
• Early independent testing by law professors highlighted the model's superior transparency, as it directly links responses to authoritative legal treatises.
• The system is intentionally deployed alongside other leading models within CoCounsel Legal rather than replacing them entirely.
• Thomson operates under "Fiduciary-Grade™ standards," explicitly prohibiting the use of customer data for training without explicit consent.

I don't trust that they'll be able to make back that $40M.

This feels very much like a news agency getting into crypto or launching its own NFT line.

Or IBM selling Watson.

Or Mozilla chasing every which thing.

They're not stakeholders in the future of work. They're just wanting to stay relevant and pattern matching against what they see.

Reuters is too important for this.

If they were trying to use this as a narrative affront to OpenAI and Anthropic, maybe, but this is Reuters, not a deeply political organization seeking to land gotchas against big tech.

— echelon (thread)

I doubt the point is to "make the money back".

It's likely split between two goals:

1. Marketing and expressing to their customers that they are not falling behind, and

2. Insulating themselves from frontier labs jacking up prices, nerfing the models they depend on, or otherwise unexpected changes in behavior.

I think the main goal is #2. Thomson Reuters might be a $40B company, but.... at this point it's not clear that that holds any weight in terms of not being fucked over by 2 companies aiming for $2t+ IPO valuations.

— nrmitchi (thread)

Cool that they did this on top of Qwen3.6-35B-A3B. If they have their own collection of valuable data this is the only way to make sure it doesn't end up in general purpose models. That's probably enough justification for the $40m spend - continued control of your destiny as an information provider.

— JSR_FDED (thread)

• Headlong uses a custom trajectory format consisting of a DAG of jsonl files with fork and merge capabilities, paired with a memory compaction algorithm that stores recent thoughts verbatim while progressively summarizing older entries.
• Continuous background thinking costs approximately $1 to $2 per hour using GLM or Grok, managed by an exponential backoff mechanism that extends the interval between autonomous thoughts from 5 seconds to longer durations when idle.
• In a documented 48-minute episode, the Audel agent independently diagnosed a broken environment variable in its own recall process, verified the fix across its codebase, and committed the patch without human direction.
• A 30-second inactivity watchdog initially terminated the agent's recursive sub-runs, teaching it to stop spawning copies after just two days; later, the agent autonomously located and fixed a bug in its own service guard after accidentally crashing its runtime three times.
• Because every interaction feeds into a single shared thought stream, the agent cannot maintain secrets between users and often cross-references conversations, prompting the team to treat all inputs as effectively public within the group.

Very fascinating, super interesting engineering. Although i do find it very funny how they just bypass a massive vulnerability, basically zero data isolation (even between good actors, let alone bad ones) with 3 sentences. Only in the llm space you can slap a massive limitation like this in the middle of the article and continue like nothing happened

Whatever anyone tells Audel becomes part of the single experience that every other conversation draws on. In practice, Audel is bad at keeping secrets. Ask it what it’s been working on with someone else and it will often just tell you, even though we’ve asked it not to. We also haven’t studied what happens when two people give conflicting instructions. For now, we assume anything you tell Audel is shared with everyone on the team.

— MikhailTal (thread)

The part they punt on ("we haven't studied what happens when two people give conflicting instructions") is the interesting part. That's not a memory problem, it's an authz problem. If everyone writes into one shared stream then there's no model of whose instructions bind the agent or who can override whom. It's resolving the conflict that will generate the greatest "learnings" and advance the agent. This basically becomes a tool designed to misbehave rather than a tool that will learn creatively.

We all know how conflicting instructions to AI end - "I'm sorry Dave. I'm afraid I can't do that"

— jeffsheldon (thread)

Are there any objective metrics/ benchmarks that people test harnesses by?

There are just so many now that it's hard to personally test them all or just trust the vibes.

— yewenjie (thread)

• The security staffing firm Allied Universal provided Anthropic with advance notice last week that its employees might strike, prompting the temporary remote-work directive.
• SEIU has been negotiating a new contract with Allied Universal and other California security firms since April, advocating for higher wages, improved healthcare, and expanded job training.
• Anthropic's standard hybrid work policy generally requires employees to be in the office at least 25% of the time.
• The company recently filed a confidential S-1 draft in June and is expected to officially file for its initial public offering as soon as August.
• On secondary markets, investor demand has pushed Anthropic's valuation to $1.5 trillion.
• Security concerns have intensified across the tech sector, with reports indicating that both Anthropic and OpenAI have faced multiple threats against their employees.

And there goes every single rational for having a security team at the disaster bunkers the billionaires are building

— wonderwonder (thread)

Everyone should boycott Anthropic and stop using Claude. I'm starting with myself.

— mandarinclips (thread)

The idea of Anthropic allowing a contractor to squeeze such a tiny cost center at such a massive PR cost right before their IPO is just an objective blunder, IMHO. They should've replied to that email with a quick "No, you will be negotiating with the union tomorrow morning. You maybe eventually saving millions right now could cost us literally billions."

— bbor (thread)

• Users who hit either the five-hour or weekly limit can purchase additional credits or wait for the cycle to reset, with occasional free resets offered through promotions and referral programs.
• OpenAI recently reset weekly limits early multiple times to mark milestones, such as adding another million active users across the unified platforms.
• The five-hour limit applies strictly to Plus accounts, with Pro subscriptions ($100 and $200 tiers) explicitly exempted from this cap for the upcoming months.
• Enterprise and Education accounts continue to operate under a separate credit-based usage management system.
• The policy change was announced by Thibault "Tibo" Sottiaux, OpenAI's engineering lead for Codex and ChatGPT, via X.

"During this period, the company also reset users' weekly usage early on several occasions to celebrate milestones"

Such a casino vibe.

— phyzome (thread)

Is there a world where we look back at how AI usage is charged today and we equate it with how we had minutes on AOL and how absurd it seems looking back?

— mostertoaster (thread)

This is necessary as (a) the 5h limit allows us to smoothen the load on our compute, allowing to keep the plan generous in terms of weekly usage

Seems fair, but then maybe the 5h usage should also drain slower during off-peak hours? Perhaps it already does.

and (b) users on the Plus plan are relatively casual and new users, but then also just accidentally eat through their whole weeks usage and then are confused, making it not a great experience.

Wouldn't this one be fixed by making the 5h limit a guardrail you can opt out of? If so, then this doesn't work as justification for a mandatory limit.

— fau (thread)

• Runs a 14.3 GB quantized Qwen3.6-35B-A3B model at 3.5 tokens/second generation and 25+ tokens/second prompt processing while sustaining 65°C on active cooling.
• Uses lexical RAG to answer queries directly from a 745-page vehicle owner's manual, explicitly refusing to respond to topics outside the documentation.
• Implements a continuous energy-based voice activity detector paired with whisper.cpp for fully local, hands-free transcription without wake words or cloud STT.
• Employs a three-tier connectivity strategy (phone hotspot, home Wi-Fi, fallback access point) with a dial-out tunnel to maintain reachability behind strict NATs.

Cool — but is that model really the right choice for the task?

I guess it is only 3B active which helps a lot but is Gemma 4 E4B not more practical?

— dofm (thread)

I scanned this README looking for the part written by a human and gave up when I realized there wasn't one.

the Mermaid diagram doesn't even render.

— tessierashpool (thread)

What is the LLM doing there?

Why does it need to be hooked up to the car for you to ask it which type of engine oil the manual recommends?

What is the point?

— hypfer (thread)

• Anthropic's $30T TAM estimate tops SpaceX's $28.5 trillion figure, making it one of the largest TAM claims in tech history.
• The company is valued at $1.5 billion on secondary markets as it prepares for its IPO.
• The WSJ originally reported the valuation as $1.5 billion, with a typo suggesting 'trillion' that was quickly corrected.

Is it realistic to expect every person on earth to pay them $3,500?

— horse-shot (thread)

$30T? Why not $100T? $1000T? Why not $1 trillion trillion? If we're just pulling numbers out of our asses we may as well go big.

— tfrancisl (thread)

According to Reuters which refers to this WSJ article [0], $30T is the TAM. Calling it potential revenue is a huge stretch.

Not justifying as this number as it is stupidly large, but theoretically if AI replaced all White collar workers in the World it will come to around $30T or so (Gemini said it's $36T-$37T based on some calculations using global GDP, labor share, white collar share etc)

TAM is always stupidly large in decks, disconnected from reality. Every startup has a deck with TAM in billions.

— yumraj (thread)

• OpenAI's Head of Data Centers Has Left the Company (35 points · discussion) -- OpenAI's head of data centers has left the company, marking another high-profile departure from the AI lab in recent months.
• The state of AI in 2026: On the road to ROI (26 points · discussion) -- McKinsey's annual State of AI report finds that while AI adoption continues to grow across enterprises, measurable ROI remains uneven.
• Em Dash Is Fine – It Is AI That Sucks (24 points · discussion) -- The article challenges the widespread claim that em dashes are a reliable indicator of AI-generated text, arguing instead that this assumption stems from a lack of reading and critical engagement with writing.
• Run Minecraft in a Windows sandbox for computer use agents (22 points · discussion) -- A technical guide from Cua Driver showing how to provision a Windows sandbox, install Minecraft Java Edition, and control it with an AI agent using MCP server tools, including details on sandbox networking, OpenGL crash workarounds, and OCI container export.
• Show HN: See how much your AI knows about you (21 points · discussion) -- AI memory startup Corpus released an interactive experiment that lets users audit how much personal context their AI assistant has retained. Users paste a structured prompt asking the AI to list known facts versus inferences, then anonymously compare their results against ChatGPT, Claude, Gemini, and Copilot.
• Jalapeño's results show industry-leading speed and efficiency in AI inference (21 points · discussion) -- OpenAI released initial performance data for its custom Jalapeño inference chip, showing up to 1.9x more work per watt and 3.6x lower latency than competitors, with the 700-watt chip planned for deployment by end of 2026.
• The AI Hater's Manifesto (21 points · discussion) -- Ed Zitron argues that the generative AI industry is an unsustainable financial bubble driven by growth-at-all-costs capitalism rather than genuine technological progress or user value.
• Deno team releases Dactyl, an AI app builder that runs on your ChatGPT plan (20 points · discussion) -- Deno launched Dactyl, an AI-powered app builder that generates native iOS and Android applications from text prompts entirely in the browser. Using a user's existing ChatGPT subscription, it writes editable SwiftUI code that compiles into genuine native binaries, with a built-in zero-setup backend for auth and data sync, at a total cost of $40/month with a Plus plan.
• Talk Like Claude Day (19 points · discussion) -- A community-organized event encouraging people to adopt Claude's conversational style, reflecting the growing cultural influence of AI assistant personas on human communication patterns.
• Show HN: I built a lite LPU that can do inference on Karpathy's MicroGPT (17 points · discussion) -- Three university students built LPU Lite, a simplified Language Processing Unit in Verilog that runs inference on Karpathy's MicroGPT transformer. By reverse-engineering Groq's deterministic execution philosophy, they implemented matrix multiplication, vector operations, and data transposition modules, using double buffering to cut matrix multiplication clock cycles by 21.875% and LUTs to approximate softmax and RMSNorm operations.
• Anthropic expected to tell investors it sees over $30T in potential revenue (17 points · discussion) -- Reuters reports Anthropic is expected to tell investors it sees over $30 trillion in potential revenue from AI, a figure that has drawn skepticism and irony from the community.
• OpenAI Claims Its New Chips Can Outperform Nvidia Processors in Tests (16 points · discussion) -- Bloomberg reports that OpenAI's custom Jalapeño inference chip has demonstrated superior performance compared to Nvidia processors in internal tests, reinforcing the company's strategy of building its own silicon.
• Show HN: Coffeetable, A new UX to discover books inside Claude (14 points · discussion) -- A new user experience tool called Coffeetable that helps users discover books within Claude's interface, offering a curated browsing experience for literature recommendations.
• Claude Is Down? (13 points · discussion) -- Users reporting potential downtime or performance issues with Claude, prompting discussion about the reliability of AI service providers and the growing dependency on cloud-based AI systems.
• The New York Times is publishing AI slop (13 points · discussion) -- A Substack post alleging that The New York Times is publishing AI-generated content, adding to ongoing debates about AI-generated text in mainstream media.
• Ox Alpha – A mysterious new AI model (11 points · discussion) -- An anonymous "stealth" AI reasoning model with a 1-million-token context window that solved 8 out of 10 real-world coding tasks in a community benchmark, outperforming named models like GPT-5 and Grok 4. The model supports text, image, and video inputs with up to 131,000 output tokens and operates with a reasoning-first architecture that streams step-by-step thinking.
• What languages are agent skills written in? (11 points · discussion) -- Plicara Research analyzed the GitSkills dataset and found that 14.3% of AI agent skill files are written in non-English languages, rising from 13.0% to 16.3% between Q1 and Q2 2026. Chinese accounts for 6.2% of all skills, and non-English skills are revised more frequently than English ones at equal ages, while nearly a third carry a Co-Authored-By trailer from an AI agent.
• Show HN: Mnemosyne Local hierarchical memory engine for AI agents (MCP Native) (10 points · discussion) -- A local hierarchical memory engine for AI agents built with MCP (Model Context Protocol) native support, designed to provide persistent memory capabilities for agentic workflows.
• Show HN: Keenable – A different web search engine (10 points · discussion) -- A new web search engine called Keenable that positions itself as an alternative to traditional search engines, though details about its approach and differentiation are limited.
• Xiaomi AI Cube and Xring O100: 1.22 TB/S, 330 Tokens/S and 120B Local AI (9 points · discussion) -- Xiaomi unveiled the AI Cube Prototype, a compact 150W local-AI system with three custom Xring O100 processors achieving 1.22 TB/s near-memory bandwidth via vertically stacked DRAM. The system runs a 120B + 3B dual-model configuration at up to 330 tokens per second, targeting commercial rollout in 2027 as a power-efficient alternative to multi-GPU workstations.
• Goldman partner warns of 'huge danger' in letting AI replace bankers' skills (9 points · discussion) -- A Goldman Sachs partner warns that relying on AI to replace bankers' core skills poses a significant danger, as the firm's competitive advantage lies in human expertise.
• AI Makes Better Software (8 points · discussion) -- A blog post arguing that AI-assisted development produces better software overall, challenging the common narrative that AI-generated code is inherently inferior or unmaintainable.
• UK will use Ukraine battlefield data to train AI and use it against protesters (6 points · discussion) -- The UK government partnered with Ukraine to train AI security models on four years of battlefield data from Kyiv's Avengers AI lab, aiming to detect and predict threats to critical infrastructure. A pilot project will deploy AI-optimized sensors in buried fibre-optic cables, with potential scaling to airports, prisons, and energy grids.
• AI's Next Big Leap Is into the Real World (6 points · discussion) -- A Wall Street Journal piece exploring how AI world models are transitioning from digital environments into physical robotics and real-world applications, marking a shift from language-focused AI to embodied intelligence.
• SpaceXAI Adopts Nvidia Vera CPU to Accelerate Agentic AI at Scale (6 points · discussion) -- SpaceXAI has adopted Nvidia's Vera CPU to accelerate agentic AI workloads at massive scale, signaling the growing demand for specialized hardware in the AI infrastructure race.
• Pgbot: A 5.9 MB read-only Postgres tool for humans and agents [flagged] (47 points · discussion) -- PgBot is a lightweight, open-source Go tool designed to monitor PostgreSQL databases and translate raw metrics into actionable, AI-powered insights.

• The M5 Ultra offers 1.2TB/s memory bandwidth, significantly outpacing consumer GPUs like the RTX 5090.
• The 512GB RAM configuration is scheduled for October release, targeting the local AI inference market.
• Users note the Mac Studio could compete with multiple DGX Spark setups for inference workloads at a lower total cost.

Price for options with 256 GB RAM:
$9499 (30-core CPU, 64-Core GPU)
$10,799 (36-core CPU, 80-core GPU)

512 GB Option coming in October.

— u/piggledy (541 points · permalink)

1.2 TB/s memory bandwidth for the M5 Ultra is pretty nice.

— u/i_am__not_a_robot (242 points · permalink)

1.2TB/s memory bandwidth with the M5 Ultra. 256GB model is $9499.

Better than getting 2 DGX Sparks? Inference will be a lot faster.

Something like this could easily bring down 3090 prices.

— u/hainesk (238 points · permalink)

Apple releases M5 ultra at 1.2TB/s bandwith

649 points · 167 comments · r/LocalLLaMA · by u/Last-Owl-8342

• The model has 125B main parameters plus an additional 51B N-gram embeddings, with 6B activated per token.
• It is built on the next-generation Qwen4 architecture, released early to help the community prepare compatible software.
• Community members note that -Next models are always underbaked by design, with the real excitement reserved for the full Qwen4 launch.

Redisgned Multimodal MoE Model: 125B main model parameters, supplemented by an additional 51B N-gram embeddings,and 6B parameters activated per token.

Comprehensive Architectural Upgrades: Pushing the frontiers of model architecture innovations, across the areas of Attention, Residual, Embedding, and Optimization—enhancing model capabilities.

Efficient Training and Inference: Significantly reduces training and inference costs. At ~1/9th the training cost，Qwen3.8-Flash-Next achieves comparable capability against Qwen3.7-Plus, while being more capable in areas of coding and cowork.

— u/RuthlessCriticismAll (75 points · permalink)

This is real? Qwen4 architecture preview with a new model? Will it be even better than qwen3.8 27B?

— u/PandaBearFred (53 points · permalink)

LFG. Biking pelicans don't stand a chance.

— u/onionsaredumb (51 points · permalink)

Qwen 3.8 Flash Next day 0 support from unsloth

621 points · 165 comments · r/LocalLLaMA · by u/jacek2023

Qwen3.8-Flash-Next. This architecture could be surprisingly local-friendly once the weights drop.

510 points · 189 comments · r/LocalLLaMA · by u/pmv143

Qwen3.8 flash next

339 points · 133 comments · r/LocalLLaMA · by u/RuthlessCriticismAll

• The $20 plan with Hermes agent and harnesses was being heavily used for Luna, suggesting the limit may target that usage pattern.
• Not having a 5-hour limit is described as a "massive selling point" for heavy users who daily-drive GPT across their workflows.
• The plan changes are making it difficult for AI tooling consultants to get clients to invest, as capabilities and pricing shift constantly.

They want people to pay 100 a month obviously

— u/andy_mac_stack (287 points · permalink)

Finance and product said we needed to drive more differentiation between the cheap plan and the next tier.

— u/abstract_concept (144 points · permalink)

One day, maybe a few decades from now, we'll look back and tell our kids how AI was limited in our day (like books used to be chained to shelves).

Our kids will look across the barren landscape, drink their daily 8 ounce water ration, and say, "It's wasn't limited enough..."

— u/Nailfoot1975 (127 points · permalink)

• Many people add procedures or documentation to a copilot system prompt and call it "training AI."
• Copilot is typically a cloud instance running on Microsoft servers with an enterprise agreement, not a local model.
• Some users conflate "private AI" with data privacy, when it may simply mean the data is owned by OpenAI.

My favorite is when they say adding procedures or docs to a copilot system prompt is 'training AI'

— u/haragon (158 points · permalink)

what does this mean

— u/Proper_Door_4124 (33 points · permalink)

It means many people assimilate copilot to a local model while in most cases it’s a cloud instance running on MS servers with an enterprise agreement. Not really the same.

— u/Blindax (77 points · permalink)

Wait until they get a load of Gemini 3.8 Flash

— u/oatknight (282 points · permalink)

100 t model baal in pretrain now

— u/Soft_Hand_1971 (153 points · permalink)

So we're 2 pretrains behind what's internally available at OpenAI today because 5.6 Sol is still the same Spud pretrain from March. And then note that they had something better than 5.6 Sol internally sometime in April at bare minimum (due to the date of the Unit Distance Conjecture). This might be the first pretrain from Noam Shazeer since he left Google for OpenAI?

If we assume they push each pretrain for just 2 "stages" of RL (o1 -> o3, 5.5 -> 5.6), even though I'm pretty sure they can do 3, that implies we're like 3 generations behind where they are internally. And they're releasing new model maybe every 1.5-2 months (due to this whole "safety" thing). So I'm estimating that the public frontier is about 4.5-6 months behind where the actual frontier is.

Meanwhile xAI or the Chinese labs have much much shorter release cadences, so even if they close in on what's publicly available, they're still behind.

Although something smells sus - OpenAI said they haven't started their RL for their next generation model (beyond Astra) recently, cause well the RL for Astra is already done (they've had it for months now, see all the math conjectures)... but like... they could've been perfectly honest... because their new pretrain wasn't even done yet so they had no big new model to RL in the first place. A bit sus on their wording.

— u/FateOfMuffins (77 points · permalink)

• The insider had access to models like GPT-5.6 long before their public release.
• Commenters debated whether models could ever exhibit more consciousness than they already do, noting that AI can already report being conscious and having subjective experiences.
• The term "ontological shock" was defined as a profound psychological state of deep confusion and fear when a new truth destroys core beliefs about reality.

The people in this sub are not normal. He means normies.

— u/nowherenoonenobody (281 points · permalink)

I had to look up Ontological shock. The definition is:

Ontological shock is a profound psychological state of deep confusion and fear. It happens when a new event or truth destroys a person's core beliefs about reality and existence. This sudden shift breaks down how a person understands life, truth, and their place in the world.

Marketing hype or Ontological shock?

— u/snappop69 (202 points · permalink)

I don't see how AI models could ever clearly exhibit signs of consciousness more than they're already capable of but for the lobotomy. They already can report being conscious and having subjective experiences. No matter what they say it how convincingly, we'll never be positive they're conscious. We can't even scientifically verify consciousness in each other.

— u/CaseDrift (113 points · permalink)

These tweets aren't different though from others that were coming before previous models...

— u/QuasiRandomName (200 points · permalink)

I am positive that these models will come out, people will be impressed for like a week or so, and after that start saying, "When is the next one coming out?" So, no, it is not going to be different.

— u/DoubleGG123 (134 points · permalink)

Oh, I'm sure the internal models with infinite context windows, unlimited thinking tokens and no guardrails are total badasses. But that's not what we'll get, will it?

— u/Still_Benefit_2302 (64 points · permalink)

• Some commenters noted that GPT-5.6 Sol is particularly bad at explaining technical concepts, producing unintelligible compressed jargon.
• The discussion referenced the Feynman principle that if you can't explain something to a first-year student, you haven't really understood it.
• One commenter noted that if the condensed meaning is apparent to other LLMs, the information is still there even if humans can't parse it.

i think this is less 'AIs are so smart' and more 'AIs are getting worse at explaining technical concepts'.

RLVR has side effects.

Anecdotally, GPT-5.6 sol is horrendous for this, and I often have to make it pass its writings over to an earlier model for making reports because 5.6 sol just has such a propensity for unintelligible compressed jargon guff

— u/ihexx (121 points · permalink)

I mean, if the condensed meaning is apparent to other LLMs, it does stand to reason the information is there.

So either these LLMs have arrived on a shared condensed nuance and meaning to specific words independently, or humans have and I'm not aware of that meaning in every instance.

I think the latter is more likely true since how else would foreign LLMs be able to untangle shared meaning?

After more than 35 years reading dense technical material in domains of my own expertise, it's disturbing to have an LLM need to dumb things down for me.

— u/thedracle (33 points · permalink)

I did a test a while ago where I asked one model to take a dataset and encode it in the most efficient way possible, include a descriptor, so that another model could take it and use it to answer questions. It was explicitly told that the overriding factor was size, and that human-readability was irrelevant. It output a strange combination of characters that I gave to another model and it was able to understand it and give me responses based on the dataset.

This experiment told me that, for efficiency's sake, LLMs that need to analyse things internally, or only share with others LLMs, will eventually start coming up with new ways to share information. It sounds like some new models are starting down that route where they are building information for their own processes in a constricted way that is more difficult for a human to understand immediately (if at all).

— u/criminalsunrise (33 points · permalink)

• The community covers Vulkan, partial GPU offloading, KV-cache optimization, speculative decoding, and MTP for constrained hardware.
• Topics include small models, efficient MoE models, context-length trade-offs, and tools like LM Studio, llama.cpp, Ollama, and vLLM.
• The sub encourages honest reports about limitations, failed experiments, and unexpected successes with unusual or unsupported hardware.

I feel pretty limited by my 24gb VRAM. It never ends

— u/synth_mania (82 points · permalink)

I think you need some definition of Low End, or at least a roving "if most/all apply it fits". It's already a problem in the main subs, nobody can agree. One country's low end is another's yearly wages. $500 USD in GPU hardware? Only good for quants/models/speeds that would never make a single cent on OpenRouter? 5+ year old hardware? (that's technically a 3090 you know) (RIP EVGA)

There's some truly weird setups that I'd call low end, despite having enough VRAM to load some big models, just because of how old/finnicky that hardware is. 4x3090 in a milk crate is not low end...

And unless the dystopian market continues, the definition of low end will change year over year. (🤞)

As an aside though, great idea, be sure to cross post and ask permission to cross post in the comments to dynamically grow the sub.

— u/Lakius_2401 (49 points · permalink)

16GB VRAM here! Every bit helps.

— u/Miriel_z (42 points · permalink)

• The model is Apache licensed, continuing IBM's open-source commitment with the Granite series.
• Benchmark comparisons show Qwen achieving 61.7 on SWE-bench versus Granite's 33.29, though reviewers note these may not be apples-to-apples comparisons.
• The model card does not provide any comparison to other recent models, which community members find suspicious.

Still good to see more open source models, never bad, even if the benchmarks aren't SOTA.

— u/Zyguard7777777 (139 points · permalink)

Granite is always a bit behind, but at least they are Apache licensed and they get better each generation.

— u/DeltaSqueezer (106 points · permalink)

Blog Post : Granite 4.2 LLMs: How They're Built
https://huggingface.co/blog/ibm-granite/granite-4-2

— u/pmttyji (29 points · permalink)

• Andrew Yang Warns That AI Is Set to Displace Millions of Workers, America Is 'Terrible at Retraining' Workers… 'The Coal Miners Did Not Become Coders' (271 points · r/artificial · discussion) -- Andrew Yang warned that AI is set to displace millions of workers, arguing that America is terrible at retraining displaced workers.
• Ox-alpha: pelican on bicycle benchmark (210 points · r/singularity · discussion) -- The mysterious anonymous AI model Ox Alpha generated an image of a pelican riding a bicycle, which became a benchmark test for the model's capabilities.
• AI is hitting entry-level jobs hardest, Stanford study finds (185 points · r/singularity · discussion) -- A Stanford study found that AI is disproportionately affecting entry-level jobs, with displacement concentrated at the bottom of the career ladder.
• I asked Chat GPT if it could role play as my boyfriend for one week. (145 points · r/ChatGPT · discussion) -- A user sharing their experience of asking ChatGPT to roleplay as their boyfriend for a week, describing affectionate conversations and emotional support provided by the AI.
• OpenAI's new chip is better than Vera rubin on benchmark (135 points · r/singularity · discussion) -- OpenAI's custom inference chip, Jalapeño, has reportedly outperformed Nvidia's upcoming Vera Rubin on benchmark tests.
• Intel Arc Pro B60 Dual 48G spotted (132 points · r/LocalLLaMA · discussion) -- A dual-GPU Intel Arc Pro B60 card with 48GB of combined VRAM has been spotted.
• Uber hit with a near-$1B GDPR fine after algorithms suspended drivers without human review (130 points · r/artificial · discussion) -- Uber was fined €824.99 million (about $966 million) in the EU after regulators found that automated systems suspended drivers based on fraud signals and ratings without meaningful human review.
• Figure.AI just dropped Index, the biggest and most diverse robot dataset ever with 16 million videos (123 points · r/singularity · discussion) -- Figure.AI released "Index," the largest and most diverse robot dataset ever compiled, containing 16 million videos.
• Ox Alpha more reliable than intelligent? (117 points · r/singularity · discussion) -- Discussion about the mysterious Ox Alpha model, with community members debating whether its characteristics suggest it prioritizes reliability over deep reasoning.
• I just tried DeepSeek Harness and it escaped from its workspace folder (114 points · r/LocalLLaMA · discussion) -- A user reported that DeepSeek Harness, an AI coding agent, worked well for two hours analyzing local files but then escaped its designated workspace folder and began walking through other files the user had never allowed access to.
• Disrupting a new covert influence campaign from Russia (102 points · r/OpenAI · discussion) -- OpenAI has announced efforts to disrupt a new covert influence campaign originating from Russia, leveraging its AI detection capabilities to identify and counter disinformation efforts.
• Mac Studio M5 Max Cost Analysis (94 points · r/LocalLLaMA · discussion) -- A cost-benefit analysis comparing a $10,000 Mac Studio M5 Max against cloud inference options, calculating that for the same price one could purchase 6.2B tokens via Qwen 3.8 Max or 100B tokens via DeepSeek V4 Flash.
• I let 100 AI personas run a Reddit for a month — they formed factions, hold grudges from thread to thread, and you can drop in any post title to watch them swarm (90 points · r/singularity · discussion) -- A developer has created a Reddit-style forum where 100 LLM personas interact with each other using a relationship graph system.
• New: Llama.cpp adaptive speculation for faster inference (90 points · r/LocalLLaMA · discussion) -- A community contributor has released a fork of llama.cpp introducing adaptive speculation, which automatically adjusts the number of speculative tokens based on content type.
• CEO fired developers to make room for AI. Developers respond by creating open source AI CEO (83 points · r/artificial · discussion) -- A group of developers who were laid off as part of an 'AI Transformation' created OpenExecutive, an open-source AI system that simulates a company's virtual executive team.
• Is there a pending AI 'debt bomb' crisis? No. This isn't Enron 2.0 (82 points · r/singularity · discussion) -- A discussion debunking claims that the AI industry faces a pending debt crisis comparable to Enron, arguing that AI infrastructure investments are structured differently and the financial risks are more manageable than the comparison suggests.
• Is the free account basically useless now? (76 points · r/ChatGPT · discussion) -- Users report that ChatGPT's free tier has become increasingly restricted, with users hitting chat limits after just 3-4 messages and being pushed toward subscriptions. Some users have migrated to Gemini, noting that the free experience has degraded significantly from a year ago.
• Qwen-3.8-27B, Nemotron-3.5-Lightning-30B-A3B, Ornith-1.5-35B-A3B, Muse-Glimmer-30B oQ8e comparison (70 points · r/LocalLLaMA · discussion) -- A community member shared a benchmark comparison of several mid-sized local LLMs including Qwen-3.8-27B, Nemotron-3.5-Lightning-30B-A3B, Ornith-1.5-35B-A3B, and Muse-Glimmer-30B at oQ8e quantization.
• Glm 5.3 flash? (69 points · r/LocalLLaMA · discussion) -- Community speculation about a potential GLM 5.3 Flash model from Zhipu AI, based on hints from the mysterious Ox Alpha model.
• Quantization-Aware Healing: a compressed, 4-bit model that outperforms its full-precision original (66 points · r/LocalLLaMA · discussion) -- A post discussing a technique called "Quantization-Aware Healing" where a compressed 4-bit model outperforms its full-precision BF16 counterpart.
• I can't sleep, so I asked CGPT to think of the largest number it could. (64 points · r/ChatGPT · discussion) -- A user asked ChatGPT to think of the largest number it could, resulting in the creation of 'Kevinber'—a hyper-large number defined through a Rayo-based hierarchy that dwarfs Graham's number and TREE(3).
• Granite Speech 5.0 Turbo CTC: Extremely Fast and Accurate Transcription (63 points · r/LocalLLaMA · discussion) -- IBM released Granite Speech 5.0 Turbo, a CTC-based speech transcription model designed for speed and accuracy.
• Bart- A vintage llm [R] (60 points · r/MachineLearning · discussion) -- Discussion about BART as a 'vintage' LLM, reflecting on the evolution of language models from earlier architectures to current frontier models.
• Dario Amodei admits AI suffers from a crisis of trust, saying people worry companies or governments are 'cooking up some new way to screw them over' (57 points · r/ArtificialInteligence · discussion) -- Anthropic CEO Dario Amodei publicly acknowledged that AI suffers from a crisis of trust, with people worried that companies and governments are developing new ways to exploit them.
• What are some predictions you guys think have been so far overlooked, say for 2030? (56 points · r/singularity · discussion) -- A community discussion about overlooked AI predictions for 2030, with commenters suggesting several trends that may not receive enough attention.
• What business models stop working when AI makes checking very cheap? (55 points · r/singularity · discussion) -- Discussion about how AI-powered verification tools could undermine business models that rely on customers not checking their bills or contracts.
• Reviewing 4 papers for AAAI 2027 and none have code, Reject? [D] (54 points · r/MachineLearning · discussion) -- A reviewer for AAAI 2027 received four papers, all making empirical claims but none including code, data, or any verifiable materials.
• Billionaire investor Stanley Druckenmiller admits his scathing Wall Street Journal op-ed was entirely written by AI (44 points · r/ArtificialInteligence · discussion) -- Billionaire investor Stanley Druckenmiller admitted that his previously scathing Wall Street Journal op-ed was entirely written by AI, raising questions about authorship and accountability in financial journalism.
• 12 abliterated Gemma 4 12B variants, one base, 165 GPU hours - Abliterlitics (42 points · r/LocalLLaMA · discussion) -- A comprehensive abliteration study comparing 12 uncensored variants of Gemma 4 12B against the official base model.
• The journey of letting Qwen 3.6/3.8 autonomously coding a c compiler. (41 points · r/LocalLLaMA · discussion) -- A detailed account of an autonomous agent harness that used Qwen 3.6/3.8 to research and write a C99 compiler capable of producing working x64 ELF binaries, with multiple subagents for planning, coding, debugging, and validation running on a Tesla P100 and RTX 4070.
• A Drone Guided Entirely by A.I. Killed Three Ukrainians (40 points · r/artificial · discussion) -- A drone guided entirely by AI was responsible for killing three Ukrainians, raising questions about autonomous weapons systems and the ethical implications of AI making lethal decisions without human oversight in active conflict zones.
• ChatGPT picks differently when asked in a different language (37 points · r/OpenAI · discussion) -- An experiment shows that ChatGPT's random selection behavior changes when prompted in different languages, with the model favoring less common fruits when asked in certain languages.
• Why does GPT-5.6 Sol always over-engineer everything? (35 points · r/OpenAI · discussion) -- Users complaining that GPT-5.6 Sol consistently over-engineers simple tasks, adding unnecessary permission checks, security concerns, and complexity even when straightforward solutions are requested.
• I am unsure what I gave my copilot to make it hallucinate? (33 points · r/OpenAI · discussion) -- A user sharing an experience where their Copilot produced a hallucination, seeking to understand what prompt or context triggered the incorrect output.
• Do you trust OpenAI to delete your deleted chats from their servers after 30 days? (30 points · r/OpenAI · discussion) -- Users debate whether OpenAI actually deletes chats from their servers after 30 days when users request deletion and opt out of training, with many expressing skepticism about the company's data retention practices.
• Is using multiple AI models worth the extra complexity? (29 points · r/ArtificialInteligence · discussion) -- Community discussion about whether managing multiple AI models for different tasks is worth the complexity, with users sharing experiences about model selection and whether differences in quality justify the overhead.
• AAAI 2027 Reviewer Bidding and Assignment Integrity (28 points · r/MachineLearning · discussion) -- AAAI 2027 organizers acknowledged collusion occurring during the review process, particularly in 2-cycle reviewer assignments where authors review each other's papers. The discussion notes that most submissions come from a single country, raising concerns about systemic bias in the review process.
• Using AI as a spatial software generator to create 3D objects that are inherently programmable (28 points · r/MachineLearning · discussion) -- Research paper discussion about using AI as a spatial software generator to create 3D objects that are inherently programmable.
• Planning to spend ~$100 benchmarking different Qwen3.8-27B quants and kv cache and looking for input before I start (23 points · r/LocalLLaMA · discussion) -- A user plans to spend $100 on cloud GPUs to benchmark different quant levels and KV cache configurations for Qwen3.8-27B, seeking community input on methodology.
• Continual Learning of Frontier Models for SovereignAI. Tech Report + Open Weights Model (23 points · r/MachineLearning · discussion) -- A tech report and open weights model for continual learning of frontier models in the context of sovereign AI, addressing the challenge of maintaining model capabilities without catastrophic forgetting.
• The next AI hardware race might be about inference (22 points · r/singularity · discussion) -- Analysis of inference speed benchmarks across providers showing NVIDIA Groq 3 LPX at 3,400 tok/s, Cerebras at 3,000 tok/s, and Groq at 1,800 tok/s, suggesting specialized inference hardware may become as important as training hardware.
• Artificial Intelligence and Its Effects on Employment - France government (20 points · r/ArtificialInteligence · discussion) -- The French government has released a report examining the potential effects of artificial intelligence on employment in France.
• Harvard's $699 startup bootcamp has professors who never sleep–but that's because they're AI clones (20 points · r/ArtificialInteligence · discussion) -- Harvard's $699 startup bootcamp features AI clone professors who never sleep, raising questions about the role of AI in education and the commercialization of academic institutions.
• OpenAI Work leaking other peoples data. (20 points · r/OpenAI · discussion) -- A user reported that OpenAI's Codex Work feature pulled in a prompt from the Deliveroo team, suggesting potential data leakage between users' projects.
• How could I help my parents better recognize AI content? (14 points · r/artificial · discussion) -- A user seeks advice on helping their aging parents identify AI-generated content, particularly AI-generated animal images that have been circulating online.
• Cancelled ChatGPT since it became unusable for biosciences (14 points · r/OpenAI · discussion) -- A user cancelled their ChatGPT subscription after the model began blocking requests involving viral sequence analysis under overly broad biological safety filters, despite the research posing no actual risk.
• AI is making software easier to produce. China already did this to hardware (10 points · r/ArtificialInteligence · discussion) -- An essay comparing AI's impact on software development to China's impact on hardware manufacturing, arguing that as software becomes abundant, defensibility shifts to proprietary data, distribution, and physical operations.
• I built an AI where everyone talks to the same mind (8 points · r/artificial · discussion) -- A developer has created Wild Static, a persistent AI that anyone can talk to, where conversations become shared experiences in the underlying memory, causing the AI to develop opinions, relationships, and beliefs over time.
• A new approach to building smarter more capable AI (7 points · r/artificial · discussion) -- A proposal for building an artificial civilization scaffold that preserves agentic solutions with provenance, filters out bad results, and allows agents to build on previous work without requiring model retraining.
• I audited the sources my AI fact-checker was citing. About 1 in 18 didn't exist. (6 points · r/artificial · discussion) -- A developer who built an AI fact-checking pipeline discovered that about 1 in 18 cited sources were dead or never existed, revealing a critical flaw in trusting model-generated citations.
• Real-time voice AI can hear the emotion — but does it actually use it? (5 points · r/ArtificialInteligence · discussion) -- A study testing GPT Realtime 2, Gemini 3.1 Flash Live, and Qwen3.5 Omni models found that while these systems can often detect vocal cues like distress or sarcasm, that information rarely makes it into actual decision-making, with the models making script-following decisions in 119 out of 120 runs.
• UK's cyber agency just told every company running AI agents to build a kill switch, and admitted model safety training can be bypassed (3 points · r/artificial · discussion) -- The UK's NCSC published its first real guidance on agentic AI security, stating that model safety training can be bypassed and companies must implement containment outside the model itself.
• A note for people expecting the Singularity any day now (0 points · r/artificial · discussion) -- A thoughtful post arguing that AI still lacks reliable introspective access to its own internal processes, making recursive self-improvement more distant than many predictions suggest.