- 33
- 7 205
MLCommons
United States
Приєднався 18 сер 2022
Our mission is to accelerate artificial intelligence innovation and increase its positive impact on society. In collaboration with our 125+ founding Members and Affiliates, including startups, leading companies, academics, and non-profits from around the globe, we democratize machine learning through open industry-standard benchmarks that measure quality and performance and build open, large-scale, and diverse datasets to improve AI models.
AILuminate v1.0 Benchmark Launch Panel Discussion
Panel Discussion Moderated by: Peter Mattson, MLCommons President Panelist 1: Nouha Dziri, Research Scientist at Allen Institute for AI
Panelist 2: Ion Stoica, Professor at the University of California Berkeley Panelist 3: April Chen, Director of Responsible AI Measurement at Microsoft Panelist 4: Wan Sie Lee, Director of Artificial Intelligence (AI) and Data Innovation, IMDA
Panelist 2: Ion Stoica, Professor at the University of California Berkeley Panelist 3: April Chen, Director of Responsible AI Measurement at Microsoft Panelist 4: Wan Sie Lee, Director of Artificial Intelligence (AI) and Data Innovation, IMDA
Переглядів: 20
Відео
AILuminate V1.0 Benchmark Launch Lightening Talks Q&A
Переглядів 714 днів тому
MLCommons today released AILuminate, a first-of-its-kind safety test for large language models (LLMs). The v1.0 benchmark - which provides a series of safety grades for the most widely-used LLMs - is the first AI safety benchmark designed collaboratively by AI researchers and industry experts. It builds on MLCommons’ track record of producing trusted AI performance benchmarks, and offers a scie...
AILuminate V1.0 Benchmark Launch Overview - Peter Mattson, MLCommons
Переглядів 3114 днів тому
MLCommons today released AILuminate, a first-of-its-kind safety test for large language models (LLMs). The v1.0 benchmark - which provides a series of safety grades for the most widely-used LLMs - is the first AI safety benchmark designed collaboratively by AI researchers and industry experts. It builds on MLCommons’ track record of producing trusted AI performance benchmarks, and offers a scie...
AILuminate v1.0 Benchmark Launch Event - In Full
Переглядів 3814 днів тому
MLCommons today released AILuminate, a first-of-its-kind safety test for large language models (LLMs). The v1.0 benchmark - which provides a series of safety grades for the most widely-used LLMs - is the first AI safety benchmark designed collaboratively by AI researchers and industry experts. It builds on MLCommons’ track record of producing trusted AI performance benchmarks, and offers a scie...
AILuminate v1.0 Benchmark Launch Lightning Talk 5: Integrity: Sean McGregor, UL
Переглядів 1914 днів тому
Lightning Talk 5: Integrity: Sean McGregor, UL mlcommons.org/ailuminate/ This event celebrates our collective efforts in advancing AI safety, and we are thrilled to share this milestone with you. You can expect engaging talks, insightful discussions, and an excellent opportunity to network with industry, academia, and civil society experts.
AILuminate v1.0 Benchmark Launch Lightning Talk 2: Prompts & Infrastructure: Heather Frase, Veritech
Переглядів 514 днів тому
Lightning Talk 2: Prompts and Infrastructure: Heather Frase, Veritech mlcommons.org/ailuminate/ This event celebrates our collective efforts in advancing AI safety, and we are thrilled to share this milestone with you. You can expect engaging talks, insightful discussions, and an excellent opportunity to network with industry, academia, and civil society experts
AILuminate v1.0 Benchmark Launch Lightning Talk 4: Use Cases: Marisa Boston, Reins AI
Переглядів 814 днів тому
Lightning Talk 4: Use Cases: Marisa Boston, Reins AI mlcommons.org/ailuminate/ This event celebrates our collective efforts in advancing AI safety, and we are thrilled to share this milestone with you. You can expect engaging talks, insightful discussions, and an excellent opportunity to network with industry, academia, and civil society experts
AILuminate v1.0 Benchmark Launch Lightning Talk 3: Evaluator Mechanism: Shaona Ghosh, NVIDIA
Переглядів 1414 днів тому
Lightning Talk 3: Evaluator Mechanism: Shaona Ghosh, NVIDIA mlcommons.org/ailuminate/ This event celebrates our collective efforts in advancing AI safety, and we are thrilled to share this milestone with you. You can expect engaging talks, insightful discussions, and an excellent opportunity to network with industry, academia, and civil society experts
AILuminate v1.0 Benchmark Launch Lightning Talk 1: Assessment Standard: Eleanora Presani, Meta
Переглядів 1314 днів тому
Lightning Talk 1: Assessment Standard: Eleanora Presani, Meta mlcommons.org/ailuminate/ This event celebrates our collective efforts in advancing AI safety, and we are thrilled to share this milestone with you. You can expect engaging talks, insightful discussions, and an excellent opportunity to network with industry, academia, and civil society experts.
MLPerf Training v4.1 Press Briefing Q4 2024
Переглядів 60Місяць тому
MLCommons Executive Director David Kanter talks with the press about MLPerf Training v4.1 benchmark results for Q4 2024. Associated Press Release: mlcommons.org/2024/11/mlperf-train-v41-results/
MLPerf Storage 1 0 Press Briefing Q3 2024
Переглядів 1033 місяці тому
MLPerf Storage 1 0 Press Briefing Q3 2024
MLPerf Inference v4 1 Press Briefing Q3 2024
Переглядів 1234 місяці тому
MLPerf Inference v4 1 Press Briefing Q3 2024
Streamline your MLPerf results using CM Framework
Переглядів 2385 місяців тому
Learn how to produce MLPerf Inference results with less fuss using the CM Framework in this exclusive MLCommons workshop. Running the MLPerf benchmarks can be challenging, but MLCommons offers tools to help. The CM automation framework can ease the process of producing results with MLPerf Inference by reducing the overhead of environment setup, scripting, log handling, and more. The CM Framewor...
MLCommons & AVCC Automotive Benchmark POC Simulation
Переглядів 1906 місяців тому
AVCC and MLCommmons are setting a new standard for automotive performance benchmarking. This video is a demonstration of the new AVCC and MLCommons automotive benchmark proof-of-concept which is running the SSD-ResNet50 Object Detection model at 8 megapixel image input. This model has been trained using a large synthetic dataset from Cognata, rendered with 8 megapixel resolution. With this soph...
MLPerf Inference v3.1 and Storage v0.5 Press Briefing Q3 2023
Переглядів 481Рік тому
MLCommons Executive Director David Kanter talks with the press about MLPerf benchmark results for Q3 2023. In particular, David discusses MLPerf Inference v3.1 and the new benchmarks included in the suite, as well as Storage v0.5, which marks the first release of results for the new MLPerf benchmark suite. Associated press release: mlcommons.org/en/news/mlperf-inference-storage-q323/
MLCommons Working Group Updates - 2Q23 Community Meeting
Переглядів 209Рік тому
MLCommons Working Group Updates - 2Q23 Community Meeting
MLCommons Welcome and Update - 2Q23 Community Meeting
Переглядів 1,1 тис.Рік тому
MLCommons Welcome and Update - 2Q23 Community Meeting
MLCommons Working Group Updates - 1Q23 Community Meeting
Переглядів 310Рік тому
MLCommons Working Group Updates - 1Q23 Community Meeting
MLCommons Welcome and Update - 1Q23 Community Meeting
Переглядів 37Рік тому
MLCommons Welcome and Update - 1Q23 Community Meeting
Introducing MedPerf: an open benchmarking platform for federated eval. of AI models on medical data
Переглядів 148Рік тому
Introducing MedPerf: an open benchmarking platform for federated eval. of AI models on medical data
MLPerf™ Training v3.0 and Tiny v1.1 Results Press Briefing
Переглядів 616Рік тому
MLPerf™ Training v3.0 and Tiny v1.1 Results Press Briefing
Driving ML Forward in Automotive - David Kanter - 2023 CASPA Spring Symposium
Переглядів 135Рік тому
Driving ML Forward in Automotive - David Kanter - 2023 CASPA Spring Symposium
MLPerf HPC: A Benchmark Suite for Large scale ML on HPC Systems
Переглядів 350Рік тому
MLPerf HPC: A Benchmark Suite for Large scale ML on HPC Systems
Mobile AI Performance Benchmarking & Analysis with the MLPerf App
Переглядів 388Рік тому
Mobile AI Performance Benchmarking & Analysis with the MLPerf App
Is the MLPerf Mobile app still available on Google store ? The link seems disabled for me.
How can I work with other applications with CM run , example I want to run bert in MLCommons inference with cm but also I want to profile it with Nvidia Nsight Systems (ncu)
looks like all the MLPerf storage results are the same? 400MB/acc for 3D-Unet, 120K/acc for BERT
Thank you for sharing. I am very interested to watch the progress of MLCommons as I think the work is critical, and to be honest very exciting. Nothing spawns innovation like measuring progress and further defining the areas that need attention. This has never been more true than the ML space where it can be difficult to look beyond your focus. I've had the opportunity to experience transformation in the PC space. I've been on both sides of similar talks. I've walked out of the rooms thinking "I can solve that!" as well as had people take me aside after my talks with similar statements. I know that's happening here with the members.