MySql PHP 从逗号分隔的数据(标签)中选择非重复值的计数


MySql PHP select count of distinct values from comma separated data (tags)

如何从MySql中存储为逗号分隔值的数据中选择非重复值的计数?最后,我将使用PHP从MySql输出数据。

里面有什么,是每个帖子的标签。所以最后,我尝试输出数据,就像stackoverflow处理它的标签一样,像这样:

tag-name x 5

这就是表中数据的样子(对不起,内容,但它是一个食谱网站(。

"postId"    "tags"                                  "category-code"
"1"         "pho,pork"                              "1"
"2"         "fried-rice,chicken"                    "1"
"3"         "fried-rice,pork"                       "1"
"4"         "chicken-calzone,chicken"               "1"
"5"         "fettuccine,chicken"                    "1"
"6"         "spaghetti,chicken"                     "1"
"7"         "spaghetti,chorizo"                     "1"
"8"         "spaghetti,meat-balls"                  "1"
"9"         "miso-soup"                             "1"
"10"        "chanko-nabe"                           "1"
"11"        "chicken-manchurian,chicken,manchurain" "1"
"12"        "pork-manchurian,pork,manchurain"       "1"
"13"        "sweet-and-sour-pork,pork"              "1"
"14"        "peking-duck,duck"                      "1"

输出

chicken             5 // occurs 5 time in the data above
pork                4 // occurs 4 time in the data above
spaghetti           3 // an so on
fried-rice          2
manchurian          2
pho                 1
chicken-calzone     1
fettuccine          1
chorizo             1
meat-balls          1
miso-soup           1
chanko-nabe         1
chicken-manchurian  1
pork-manchurian     1
sweet-n-sour-pork   1
peking-duck         1
duck                1

我正在尝试select count of all distinct values in there,但由于它是逗号分隔的数据,因此似乎没有办法做到这一点。 select distinct行不通。

你能想到一个好的方法来使用 mysql 或使用 php 来获取输出,就像我所做的那样吗?

解决方案

我真的不知道如何在不创建包含数字的表的情况下将逗号分隔值的水平列表转换为行列表,与逗号分隔值一样多的数字。如果你可以创建这个表,这是我的答案:

SELECT 
  SUBSTRING_INDEX(SUBSTRING_INDEX(all_tags, ',', num), ',', -1) AS one_tag,
  COUNT(*) AS cnt
FROM (
  SELECT
    GROUP_CONCAT(tags separator ',') AS all_tags,
    LENGTH(GROUP_CONCAT(tags SEPARATOR ',')) - LENGTH(REPLACE(GROUP_CONCAT(tags SEPARATOR ','), ',', '')) + 1 AS count_tags
  FROM test
) t
JOIN numbers n
ON n.num <= t.count_tags
GROUP BY one_tag
ORDER BY cnt DESC;

返回:

+---------------------+-----+
| one_tag             | cnt |
+---------------------+-----+
| chicken             |   5 |
| pork                |   4 |
| spaghetti           |   3 |
| fried-rice          |   2 |
| manchurain          |   2 |
| pho                 |   1 |
| chicken-calzone     |   1 |
| fettuccine          |   1 |
| chorizo             |   1 |
| meat-balls          |   1 |
| miso-soup           |   1 |
| chanko-nabe         |   1 |
| chicken-manchurian  |   1 |
| pork-manchurian     |   1 |
| sweet-and-sour-pork |   1 |
| peking-duck         |   1 |
| duck                |   1 |
+---------------------+-----+
17 rows in set (0.01 sec)

参见 sqlfiddle


解释

场景

  1. 我们使用逗号连接所有标签,只创建一个标签列表,而不是每行一个
  2. 我们计算列表中有多少标签
  3. 我们找到如何在此列表中获取一个值
  4. 我们找到了如何将所有值获取为不同的行
  5. 我们按其值分组计算标签

上下文

让我们构建您的架构:

CREATE TABLE test (
    id INT PRIMARY KEY,
    tags VARCHAR(255)
);
INSERT INTO test VALUES
    ("1",         "pho,pork"),
    ("2",         "fried-rice,chicken"),
    ("3",         "fried-rice,pork"),
    ("4",         "chicken-calzone,chicken"),
    ("5",         "fettuccine,chicken"),
    ("6",         "spaghetti,chicken"),
    ("7",         "spaghetti,chorizo"),
    ("8",         "spaghetti,meat-balls"),
    ("9",         "miso-soup"),
    ("10",        "chanko-nabe"),
    ("11",        "chicken-manchurian,chicken,manchurain"),
    ("12",        "pork-manchurian,pork,manchurain"),
    ("13",        "sweet-and-sour-pork,pork"),
    ("14",        "peking-duck,duck");

连接所有标签列表

我们将在一行中处理所有标签,因此我们使用GROUP_CONCAT来完成这项工作:

SELECT GROUP_CONCAT(tags SEPARATOR ',') FROM test;

返回以逗号分隔的所有标记:

河粉,猪肉,炒饭,鸡肉,

炒饭,猪肉,鸡肉饺子

,鸡肉,意大利细面条,鸡肉,意大利面条,香肠,意大利面条,肉丸,味噌汤,火锅,满洲鸡,鸡肉,满洲,满洲猪肉,猪肉,满洲,糖醋猪肉,猪肉,北京烤鸭,鸭

统计所有标签

要计算所有标签,我们得到完整标签列表的长度,并在将,替换为零后删除完整标签列表的长度。我们加 1,因为分隔符位于两个值之间。

SELECT LENGTH(GROUP_CONCAT(tags SEPARATOR ',')) - LENGTH(REPLACE(GROUP_CONCAT(tags SEPARATOR ','), ',', '')) + 1 AS count_tags
FROM test;

返回:

+------------+
| count_tags |
+------------+
|         28 |
+------------+
1 row in set (0.00 sec)

获取标签列表中的第 N 个标签

我们使用SUBSTRING_INDEX函数来获取

-- returns the string until the 2nd delimiter''s occurrence from left to right: a,b
SELECT SUBSTRING_INDEX('a,b,c', ',', 2);
-- return the string until the 1st delimiter, from right to left: c
SELECT SUBSTRING_INDEX('a,b,c', ',', -1);
-- we need both to get: b (with 2 being the tag number)
SELECT SUBSTRING_INDEX(SUBSTRING_INDEX('a,b,c', ',', 2), ',', -1);

有了这样的逻辑,要获取列表中的第三个标签,我们使用:

SELECT SUBSTRING_INDEX(SUBSTRING_INDEX(GROUP_CONCAT(tags SEPARATOR ','), ',', 3), ',', -1)
FROM test;

返回:

+-------------------------------------------------------------------------------------+
| SUBSTRING_INDEX(SUBSTRING_INDEX(GROUP_CONCAT(tags SEPARATOR ','), ',', 3), ',', -1) |
+-------------------------------------------------------------------------------------+
| fried-rice                                                                          |
+-------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

获取不同行形式的所有值

我的想法有点棘手:

  1. 我知道我们可以通过连接表来创建行
  2. 我需要使用上面的请求获取列表中的第 N 个标签

因此,我们将创建一个表,其中包含从 1 到列表中可能拥有的最大标签数的所有数字。如果可以有 1M 值,请创建从 1 到 1,000,000 的 1M 条目。对于 100 个标签,这将是:

CREATE TABLE numbers (
  num INT PRIMARY KEY
);
INSERT INTO numbers VALUES
    ( 1 ), ( 2 ), ( 3 ), ( 4 ), ( 5 ), ( 6 ), ( 7 ), ( 8 ), ( 9 ), ( 10 ), 
    ( 11 ), ( 12 ), ( 13 ), ( 14 ), ( 15 ), ( 16 ), ( 17 ), ( 18 ), ( 19 ), ( 20 ), 
    ( 21 ), ( 22 ), ( 23 ), ( 24 ), ( 25 ), ( 26 ), ( 27 ), ( 28 ), ( 29 ), ( 30 ), 
    ( 31 ), ( 32 ), ( 33 ), ( 34 ), ( 35 ), ( 36 ), ( 37 ), ( 38 ), ( 39 ), ( 40 ), 
    ( 41 ), ( 42 ), ( 43 ), ( 44 ), ( 45 ), ( 46 ), ( 47 ), ( 48 ), ( 49 ), ( 50 ), 
    ( 51 ), ( 52 ), ( 53 ), ( 54 ), ( 55 ), ( 56 ), ( 57 ), ( 58 ), ( 59 ), ( 60 ), 
    ( 61 ), ( 62 ), ( 63 ), ( 64 ), ( 65 ), ( 66 ), ( 67 ), ( 68 ), ( 69 ), ( 70 ), 
    ( 71 ), ( 72 ), ( 73 ), ( 74 ), ( 75 ), ( 76 ), ( 77 ), ( 78 ), ( 79 ), ( 80 ), 
    ( 81 ), ( 82 ), ( 83 ), ( 84 ), ( 85 ), ( 86 ), ( 87 ), ( 88 ), ( 89 ), ( 90 ), 
    ( 91 ), ( 92 ), ( 93 ), ( 94 ), ( 95 ), ( 96 ), ( 97 ), ( 98 ), ( 99 ), ( 100 );

现在,我们使用以下查询获得第 num 个(num 是 number 中的一行(:

SELECT n.num, SUBSTRING_INDEX(SUBSTRING_INDEX(all_tags, ',', num), ',', -1) as one_tag
FROM (
  SELECT
    GROUP_CONCAT(tags SEPARATOR ',') AS all_tags,
    LENGTH(GROUP_CONCAT(tags SEPARATOR ',')) - LENGTH(REPLACE(GROUP_CONCAT(tags SEPARATOR ','), ',', '')) + 1 AS count_tags
  FROM test
) t
JOIN numbers n
ON n.num <= t.count_tags

返回:

+-----+---------------------+
| num | one_tag             |
+-----+---------------------+
|   1 | pho                 |
|   2 | pork                |
|   3 | fried-rice          |
|   4 | chicken             |
|   5 | fried-rice          |
|   6 | pork                |
|   7 | chicken-calzone     |
|   8 | chicken             |
|   9 | fettuccine          |
|  10 | chicken             |
|  11 | spaghetti           |
|  12 | chicken             |
|  13 | spaghetti           |
|  14 | chorizo             |
|  15 | spaghetti           |
|  16 | meat-balls          |
|  17 | miso-soup           |
|  18 | chanko-nabe         |
|  19 | chicken-manchurian  |
|  20 | chicken             |
|  21 | manchurain          |
|  22 | pork-manchurian     |
|  23 | pork                |
|  24 | manchurain          |
|  25 | sweet-and-sour-pork |
|  26 | pork                |
|  27 | peking-duck         |
|  28 | duck                |
+-----+---------------------+
28 rows in set (0.01 sec)

计算标记出现次数

一旦我们现在有了经典行,我们就可以轻松计算每个标签的出现次数。

请参阅此答案的顶部以查看请求。

Alain Tiembo有一个很好的答案,解释了下面的许多机制。但是,他的解决方案需要一个临时表(数字(来解决问题。作为后续答案,我将他的所有步骤合并到一个查询中(对原始表使用 tablename(:

    SELECT t.tags, count(*) AS occurence FROM
    (SELECT
      tablename.id,
      SUBSTRING_INDEX(SUBSTRING_INDEX(tablename.tags, ',', numbers.n), ',', -1) tags
    FROM
      (SELECT 1 n UNION ALL SELECT 2
       UNION ALL SELECT 3 UNION ALL SELECT 4) numbers INNER JOIN tablename
      ON CHAR_LENGTH(tablename.tags)
         -CHAR_LENGTH(REPLACE(tablename.tags, ',', ''))>=numbers.n-1
    ORDER BY
      id, n) t
    GROUP BY t.tags
    ORDER BY occurence DESC, t.tags ASC

有关演示目的,请参阅 SQLFiddle

首先,您应该使用连接表存储它,每个帖子和标签一行。 然而,有时我们无法控制我们正在处理的数据的结构。

假设您有一个有效标签列表,您可以执行所需的操作:

select vt.tag, count(t.postid) as cnt
from validtags vt left join
     table t
     on find_in_set(vt.tag, t.tags) > 0
group by vt.tag
order by cnt desc;

建议的方法是不要在单个列中存储多个值,而是创建一个交集表。

因此,您的表将包含以下列:
1.标签: tag_id, 名称
2. 帖子: post_id, category_code
3. int_tags_to_posts:post_id、tag_id

要获取计数,请执行以下操作:
select t.name, count(*) from tags t, posts p, int_tags_to_posts i where i.post_id = p.post_id and i.tag_id = t.tag_id group by i.tag_id order by count(*) desc;

这应该有效:

SELECT tag, count(0) count FROM (
    SELECT tOut.*, REPLACE(SUBSTRING(SUBSTRING_INDEX(tags, ',', ocur_rank), LENGTH(SUBSTRING_INDEX(tags, ',', ocur_rank - 1)) + 1), ',', '') tag
      FROM (
        SELECT @num_type := if(@id_check = tY.id, @num_type + 1, 1) AS ocur_rank, @id_check := tY.id as id_check, tY.*
          FROM (
            SELECT LENGTH(tags) - LENGTH(REPLACE(tags, ',', '')) AS num_ocur, id, tags FROM tablename
          ) tX
          INNER JOIN (SELECT LENGTH(tags) - LENGTH(REPLACE(tags, ',', '')) AS num_ocur, id, tags FROM tablename) tY
          INNER JOIN (SELECT @num_type := 0, @id_check := 'some_id') tZ
       ) tOut
     WHERE ocur_rank <= num_ocur + 1
) tempTable GROUP BY tag ORDER BY count DESC;

将"表名"替换为表的名称。

这个答案来自杰西·佩林(Jesse Perring(在此页面上发布的解决方案:

http://dev.mysql.com/doc/refman/5.0/en/string-functions.html#c12113